Sep 17 13:53:58.893647 ip-10-0-130-163 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 13:53:58.893660 ip-10-0-130-163 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 13:53:58.893670 ip-10-0-130-163 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 13:53:58.893992 ip-10-0-130-163 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 13:54:08.906711 ip-10-0-130-163 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 13:54:08.906728 ip-10-0-130-163 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 04339c3c009247eeb55f40790019c752 --
Sep 17 13:55:56.405989 ip-10-0-130-163 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 13:55:56.790070 ip-10-0-130-163 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 13:55:56.790070 ip-10-0-130-163 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 13:55:56.790070 ip-10-0-130-163 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 13:55:56.790070 ip-10-0-130-163 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 13:55:56.790070 ip-10-0-130-163 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 13:55:56.791153 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.791076    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 13:55:56.795007 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.794985    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.795007 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795001    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.795007 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795005    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.795007 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795008    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.795007 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795013    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795018    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795021    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795025    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795028    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795030    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795034    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795036    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795039    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795041    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795044    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795047    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795049    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795052    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795055    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795057    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795060    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795063    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795065    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.795198 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795068    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795070    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795073    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795075    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795078    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795080    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795083    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795086    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795091    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795094    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795096    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795099    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795101    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795103    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795106    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795110    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795113    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795116    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795118    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795120    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.795647 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795123    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795126    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795128    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795130    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795133    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795136    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795138    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795141    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795143    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795146    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795148    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795151    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795153    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795156    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795158    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795162    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795166    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795169    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795172    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795174    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.796148 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795177    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795180    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795183    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795187    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795190    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795192    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795195    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795198    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795200    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795203    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795206    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795208    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795211    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795214    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795216    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795219    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795222    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795224    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795227    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795229    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.796622 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795235    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795238    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795241    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795244    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795246    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795249    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795251    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795254    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795256    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795259    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795261    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795264    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795266    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795269    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795271    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795274    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795981    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795988    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795992    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.797176 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795995    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.795998    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796001    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796003    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796006    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796008    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796011    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796014    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796018    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796022    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796025    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796028    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796031    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796034    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796037    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796040    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796044    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796047    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796050    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.797673 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796054    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796056    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796059    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796061    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796064    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796098    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796102    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796105    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796109    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796112    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796115    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796118    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796120    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796123    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796126    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796128    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796131    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796134    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796137    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.798158 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796139    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796142    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796144    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796147    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796149    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796152    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796154    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796157    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796160    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796164    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796168    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796171    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796173    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796176    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796178    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796181    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796183    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796186    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796188    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796191    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.798611 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796193    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796196    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796198    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796201    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796204    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796210    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796213    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796215    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796218    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796220    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796223    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796226    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796228    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796231    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796233    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796236    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796239    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796241    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796244    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796246    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.799120 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796249    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796251    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796255    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796257    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796260    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796262    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796265    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796267    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796270    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796272    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796275    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796277    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796279    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796282    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796284    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796287    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796289    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796293    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796660    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.799755 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796665    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796668    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796670    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796673    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796677    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796680    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796683    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796686    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796690    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796693    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796696    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796699    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796702    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796704    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796706    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796709    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796712    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796714    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796717    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.800209 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796720    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796722    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796725    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796727    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796745    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796748    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796751    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796753    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796756    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796759    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796761    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796766    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796769    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796772    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796774    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796777    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796779    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796782    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796784    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.800670 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796787    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796790    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796792    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796795    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796797    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796800    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796802    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796805    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796807    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796810    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796813    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796816    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796818    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796821    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796823    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796825    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796828    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796830    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796833    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796835    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.801149 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796838    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796841    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796843    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796845    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796849    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796852    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796855    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796857    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796860    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796862    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796865    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796868    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796870    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796873    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796875    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796878    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796880    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796883    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796887    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796889    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796892    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.801701 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796894    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796897    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796900    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796902    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796905    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796907    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796910    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796913    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796917    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796919    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796922    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796925    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796927    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796929    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796932    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796936    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796938    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796942    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.796945    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.802219 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797017    2570 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797049    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797055    2570 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797060    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797063    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797067    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797071    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797077    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797081    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797083    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797087    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797090    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797093    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797096    2570 flags.go:64] FLAG: --cgroup-root=""
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797098    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797101    2570 flags.go:64] FLAG: --client-ca-file=""
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797105    2570 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797107    2570 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797111    2570 flags.go:64] FLAG: --cluster-domain=""
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797114    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797117    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797120    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797123    2570 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797126    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 13:55:56.802670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797129    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797132    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797135    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797138    2570 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797142    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797145    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797149    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797152    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797155    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797158    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797161    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797164    2570 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797167    2570 flags.go:64] FLAG: --enable-server="true"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797170    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797174    2570 flags.go:64] FLAG: --event-burst="100"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797177    2570 flags.go:64] FLAG: --event-qps="50"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797180    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797183    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797186    2570 flags.go:64] FLAG: --eviction-hard=""
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797189    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797192    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797195    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797198    2570 flags.go:64] FLAG: --eviction-soft=""
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797201    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797204    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 13:55:56.803269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797207    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797210    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797213    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797215    2570 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797218    2570 flags.go:64] FLAG: --feature-gates=""
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797222    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797225    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797228    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797231    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797234    2570 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797237    2570 flags.go:64] FLAG: --help="false"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797240    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-130-163.ec2.internal"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797244    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797249    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797252    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797255    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797258    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797261    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797264    2570 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797266    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797269    2570 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797272    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797275    2570 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797277    2570 flags.go:64] FLAG: --kube-reserved=""
Sep 17 13:55:56.803926 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797280    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797283    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797286    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797288    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797291    2570 flags.go:64] FLAG: --lock-file=""
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797293    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797296    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797299    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797304    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797308    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797310    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797313    2570 flags.go:64] FLAG: --logging-format="text"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797315    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797318    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797321    2570 flags.go:64] FLAG: --manifest-url=""
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797324    2570 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797328    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797330    2570 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797335    2570 flags.go:64] FLAG: --max-pods="110"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797337    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797340    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797344    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797346    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797349    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797353    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 13:55:56.804512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797356    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797363    2570 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797366    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797368    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797371    2570 flags.go:64] FLAG: --pod-cidr=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797374    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797379    2570 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797382    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797385    2570 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797388    2570 flags.go:64] FLAG: --port="10250"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797391    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797394    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0194ea0655b0eb725"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797397    2570 flags.go:64] FLAG: --qos-reserved=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797400    2570 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797403    2570 flags.go:64] FLAG: --register-node="true"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797406    2570 flags.go:64] FLAG: --register-with-taints=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797409    2570 flags.go:64] FLAG: --registry-burst="10"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797413    2570 flags.go:64] FLAG: --registry-qps="5"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797415    2570 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797418    2570 flags.go:64] FLAG: --reserved-memory=""
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797422    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797425    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797428    2570 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797431    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797433    2570 flags.go:64] FLAG: --runonce="false"
Sep 17 13:55:56.805172 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797436    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797439    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797442    2570 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797445    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797448    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797451    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797454    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797457    2570 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797459    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797463    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797465    2570 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797468    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797471    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797473    2570 flags.go:64] FLAG: --system-cgroups=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797476    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797481    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797484    2570 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797486    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797490    2570 flags.go:64] FLAG: --tls-min-version=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797493    2570 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797495    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797498    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797501    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797504    2570 flags.go:64] FLAG: --v="2"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797507    2570 flags.go:64] FLAG: --version="false"
Sep 17 13:55:56.805814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797511    2570 flags.go:64] FLAG: --vmodule=""
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797515    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797518    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797641    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797645    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797648    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797651    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797653    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797656    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797659    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797663    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797667    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797670    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797673    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797676    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797679    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797682    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797685    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797688    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.806410 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797691    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797693    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797696    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797698    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797700    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797703    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797705    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797708    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797710    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797713    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797715    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797718    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797720    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797724    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797728    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797745    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797748    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797751    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797753    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797756    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.806894 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797758    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797762    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797765    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797769    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797771    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797774    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797777    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797779    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797782    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797784    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797787    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797789    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797792    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797794    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797797    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797799    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797802    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797804    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797807    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797809    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.807376 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797812    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797814    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797817    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797820    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797822    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797824    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797827    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797830    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797833    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797835    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797837    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797840    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797842    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797847    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797851    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797855    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797858    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797861    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797864    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797867    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.807891 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797870    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797873    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797875    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797878    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797881    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797883    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797886    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797888    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797891    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797893    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797896    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797898    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797901    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797903    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797906    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797908    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797911    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797913    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797916    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797918    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.808374 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797921    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797923    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.797926    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.797931    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.804914    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.804928    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.805935    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.805946    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806001    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806006    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806010    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806013    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806016    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806019    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806021    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.808906 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806024    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806027    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806029    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806032    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806036    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806040    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806043    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806046    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806049    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806051    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806055    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806057    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806060    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806063    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806065    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806068    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806071    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806073    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806076    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806078    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.809288 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806081    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806084    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806088    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806090    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806093    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806095    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806098    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806100    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806103    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806105    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806108    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806111    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806113    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806115    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806118    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806120    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806123    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806125    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806127    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806130    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.809782 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806133    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806135    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806137    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806140    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806142    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806145    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806147    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806150    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806152    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806154    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806157    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806159    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806161    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806164    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806167    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806170    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806173    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806176    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806178    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806180    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.810260 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806183    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806185    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806188    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806191    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806193    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806196    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806198    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806200    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806203    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806205    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806208    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806210    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806213    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806215    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806218    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806222    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806225    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806227    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806231    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.810769 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806233    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806236    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806239    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806241    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806244    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806247    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806249    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806252    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806255    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806259    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806263    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806266    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806269    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.806273    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806381    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 13:55:56.811232 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806385    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806388    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806391    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806393    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806395    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806398    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806401    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806403    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806405    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806408    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806410    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806413    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806416    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806418    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806421    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806423    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806426    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806428    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806430    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 13:55:56.811603 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806433    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806435    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806438    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806440    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806443    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806445    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806448    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806451    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806453    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806456    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806458    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806461    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806463    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806465    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806468    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806471    2570 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806473    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806475    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806478    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806480    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 13:55:56.812164 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806483    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806485    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806488    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806490    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806493    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806495    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806498    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806500    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806502    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806505    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806507    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806510    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806512    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806515    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806518    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806522    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806525    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806527    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806530    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 13:55:56.812660 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806533    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806535    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806538    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806541    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806543    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806546    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806548    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806551    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806553    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806555    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806558    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806560    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806563    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806565    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806567    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806570    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806572    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806575    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806579    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806581    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 13:55:56.813136 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806584    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806586    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806589    2570 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806591    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806593    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806596    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806598    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806601    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806603    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806605    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806608    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806611    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806613    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806616    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806619    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806623    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806626    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806628    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806631    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 13:55:56.813721 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:56.806634    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.806639    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.806761    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.809004    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.809853    2570 server.go:1019] "Starting client certificate rotation"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.809955    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.810003    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.812408    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 13:55:56.814375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.813535    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 13:55:56.824139 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.824117    2570 log.go:25] "Validated CRI v1 runtime API"
Sep 17 13:55:56.827193 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.827177    2570 log.go:25] "Validated CRI v1 image API"
Sep 17 13:55:56.827974 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.827959    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 13:55:56.830467 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.830451    2570 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cc52fa0f-d33b-41c8-9f6e-8d43dcc14d8f:/dev/nvme0n1p4 e77f0fbf-44e1-46ca-a11b-26f5cf9a09d0:/dev/nvme0n1p3]
Sep 17 13:55:56.830467 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.830468    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 13:55:56.836089 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.835980    2570 manager.go:216] Machine: {Timestamp:2026-09-17 13:55:56.833960574 +0000 UTC m=+0.317836309 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102118 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec249cfb3a84d3dea75a1d9aaab5e20a SystemUUID:ec249cfb-3a84-d3de-a75a-1d9aaab5e20a BootID:04339c3c-0092-47ee-b55f-40790019c752 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f8:a2:14:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f8:a2:14:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:ff:a2:9a:ec:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 13:55:56.836089 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.836085    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 13:55:56.836204 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.836159    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 13:55:56.839070 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.839043    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 13:55:56.839196 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.839072    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 13:55:56.839768 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.839759    2570 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 13:55:56.839800 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.839770    2570 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 13:55:56.839800 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.839777    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 13:55:56.840553 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.840543    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 13:55:56.841322 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.841300    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 13:55:56.841373 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.841346    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 13:55:56.845420 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845409    2570 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 13:55:56.845525 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845515    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 13:55:56.845557 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845546    2570 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 13:55:56.845585 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845563    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 13:55:56.845585 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845575    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 13:55:56.845585 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845585    2570 kubelet.go:398] "Adding apiserver pod source"
Sep 17 13:55:56.845716 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845602    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 13:55:56.845957 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845936    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 13:55:56.845957 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.845954    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 13:55:56.848221 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.848207    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 17 13:55:56.850594 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.850580    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 13:55:56.850639 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.850612    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 13:55:56.852238 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852226    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 13:55:56.852283 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852245    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 13:55:56.852283 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852263    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 13:55:56.852283 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852270    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 13:55:56.852283 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852278    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 13:55:56.852283 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852284    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852297    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852302    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852308    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852313    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852327    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 13:55:56.852416 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.852333    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 13:55:56.853334 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.853324    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 13:55:56.853334 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.853335    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 13:55:56.855406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.855390    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 13:55:56.855484 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.855414    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 13:55:56.855484 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.855411    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 13:55:56.857221 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.857197    2570 server.go:1268] "Started kubelet"
Sep 17 13:55:56.857340 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.857310    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 13:55:56.857432 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.857412    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 13:55:56.857509 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.857410    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 13:55:56.860582 ip-10-0-130-163 systemd[1]: Started Kubernetes Kubelet.
Sep 17 13:55:56.860927 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.860910    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 13:55:56.862578 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.862436    2570 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 13:55:56.863402 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.863378    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 13:55:56.866173 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.865407    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18d6203efe8ea5db  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-09-17 13:55:56.857173467 +0000 UTC m=+0.341049201,LastTimestamp:2026-09-17 13:55:56.857173467 +0000 UTC m=+0.341049201,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Sep 17 13:55:56.866987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.866973    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 13:55:56.867787 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.867775    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 13:55:56.867882 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.867862    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:56.868384 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868363    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 13:55:56.868384 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868385    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 13:55:56.868537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868436    2570 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 13:55:56.868537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868445    2570 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 13:55:56.868537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868462    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 13:55:56.868537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868485    2570 factory.go:55] Registering systemd factory
Sep 17 13:55:56.868537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868496    2570 factory.go:223] Registration of the systemd container factory successfully
Sep 17 13:55:56.868923 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868902    2570 factory.go:153] Registering CRI-O factory
Sep 17 13:55:56.868923 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868917    2570 factory.go:223] Registration of the crio container factory successfully
Sep 17 13:55:56.869143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868964    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 13:55:56.869143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868985    2570 factory.go:103] Registering Raw factory
Sep 17 13:55:56.869143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.868998    2570 manager.go:1196] Started watching for new ooms in manager
Sep 17 13:55:56.869370 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.869360    2570 manager.go:349] Starting recovery of all containers
Sep 17 13:55:56.876285 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.876240    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 13:55:56.878324 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.878292    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 13:55:56.878410 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.878334    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 13:55:56.880487 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.880472    2570 manager.go:354] Recovery completed
Sep 17 13:55:56.884521 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.884506    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:56.886231 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886214    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:56.886296 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886242    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:56.886296 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886252    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:56.886724 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886712    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:56.886724 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886713    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 13:55:56.886863 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886750    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 13:55:56.886863 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.886769    2570 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 13:55:56.887361 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.887347    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:56.887415 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.887376    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:56.887415 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.887388    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:56.888869 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.888855    2570 policy_none.go:49] "None policy: Start"
Sep 17 13:55:56.888908 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.888877    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 13:55:56.888908 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.888893    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 13:55:56.890900 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.890889    2570 policy_none.go:47] "Start"
Sep 17 13:55:56.892659 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.892639    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crg49"
Sep 17 13:55:56.894045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.893988    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18d6203f004a00b5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-09-17 13:55:56.886229173 +0000 UTC m=+0.370104906,LastTimestamp:2026-09-17 13:55:56.886229173 +0000 UTC m=+0.370104906,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Sep 17 13:55:56.898811 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.898750    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18d6203f004a4746  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-163.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-09-17 13:55:56.886247238 +0000 UTC m=+0.370122972,LastTimestamp:2026-09-17 13:55:56.886247238 +0000 UTC m=+0.370122972,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Sep 17 13:55:56.904108 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.904082    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crg49"
Sep 17 13:55:56.906570 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.906492    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18d6203f004a697c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-163.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-09-17 13:55:56.886255996 +0000 UTC m=+0.370131730,LastTimestamp:2026-09-17 13:55:56.886255996 +0000 UTC m=+0.370131730,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Sep 17 13:55:56.908568 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.908494    2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-163.ec2.internal.18d6203f004a00b5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18d6203f004a00b5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-09-17 13:55:56.886229173 +0000 UTC m=+0.370104906,LastTimestamp:2026-09-17 13:55:56.887359879 +0000 UTC m=+0.371235618,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.927634    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.927671    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.927689    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.927722    2570 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.928507    2570 manager.go:337] "Starting Device Plugin manager"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.928542    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.928551    2570 server.go:87] "Starting device plugin registration server"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.928846    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.928856    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.928967    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.929055    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.929064    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.929500    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:56.929560    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:56.942608 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:56.930255    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 13:55:57.028847 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.028823    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"]
Sep 17 13:55:57.028921 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.028884    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.028970 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.028927    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.029712 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029694    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029723    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029752    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029758    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029769    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029781    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.029799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.029800    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.032066 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032054    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.032214 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032201    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.032253 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032226    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.032714 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032700    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.032791 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032706    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.032791 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032721    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.032791 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032758    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.032791 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032761    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.032791 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.032774    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.034960 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.034948    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.035000 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.034970    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.035626 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.035609    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.035710 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.035638    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.035710 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.035651    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.040497 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.040445    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.040497 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.040462    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-163.ec2.internal\": node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.048257 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.048240    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-163.ec2.internal\" not found" node="ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.052552 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.052539    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-163.ec2.internal\" not found" node="ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.088020 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.087999    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.170122 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.170104    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.170184 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.170126    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f272aba07b562b95d8ca5dc3318b8a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"c4f272aba07b562b95d8ca5dc3318b8a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.170184 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.170142    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.188431 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.188414    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.270338 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270319    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.270411 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270344    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f272aba07b562b95d8ca5dc3318b8a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"c4f272aba07b562b95d8ca5dc3318b8a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.270411 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270358    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.270493 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270413    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.270493 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270423    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/352a6f4db44839b0aac72f56039c3b11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"352a6f4db44839b0aac72f56039c3b11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.270493 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.270430    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c4f272aba07b562b95d8ca5dc3318b8a-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"c4f272aba07b562b95d8ca5dc3318b8a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.288673 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.288652    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.350676 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.350630    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.355527 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.355461    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:57.389216 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.389197    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.489760 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.489723    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.590176 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.590154    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.690758 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.690692    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.791166 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.791137    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.810296 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.810280    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 13:55:57.810461 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.810445    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 13:55:57.891524 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.891491    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:57.907652 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.907617    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 13:50:58 +0000 UTC" deadline="2028-02-29 23:23:05.523352968 +0000 UTC"
Sep 17 13:55:57.907652 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.907645    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12729h27m7.615710631s"
Sep 17 13:55:57.909479 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:57.909456    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f272aba07b562b95d8ca5dc3318b8a.slice/crio-6bf6da789c80c071a21473ceafa141f8b77c858dc3eb02c80b3581fc8db3599a WatchSource:0}: Error finding container 6bf6da789c80c071a21473ceafa141f8b77c858dc3eb02c80b3581fc8db3599a: Status 404 returned error can't find the container with id 6bf6da789c80c071a21473ceafa141f8b77c858dc3eb02c80b3581fc8db3599a
Sep 17 13:55:57.915350 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.915338    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 13:55:57.931080 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.931048    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" event={"ID":"c4f272aba07b562b95d8ca5dc3318b8a","Type":"ContainerStarted","Data":"6bf6da789c80c071a21473ceafa141f8b77c858dc3eb02c80b3581fc8db3599a"}
Sep 17 13:55:57.981613 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.981555    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 13:55:57.984367 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.984352    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 13:55:57.984428 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.984382    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 13:55:57.984428 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:57.984392    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Sep 17 13:55:57.992009 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:57.991989    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:58.050028 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.050001    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 13:55:58.092904 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:55:58.092885    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Sep 17 13:55:58.152423 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.152402    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 13:55:58.168816 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.168793    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:58.181394 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.181373    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 13:55:58.182218 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.182205    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Sep 17 13:55:58.191570 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.191545    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 13:55:58.200593 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:55:58.200574    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352a6f4db44839b0aac72f56039c3b11.slice/crio-a6be05c2d4bb29f6996d97df0b96c65150a54e8eb28d2824c158dee33a84a8fb WatchSource:0}: Error finding container a6be05c2d4bb29f6996d97df0b96c65150a54e8eb28d2824c158dee33a84a8fb: Status 404 returned error can't find the container with id a6be05c2d4bb29f6996d97df0b96c65150a54e8eb28d2824c158dee33a84a8fb
Sep 17 13:55:58.343718 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:55:58.343643    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 13:56:00.094145 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.094100    2570 apiserver.go:52] "Watching apiserver"
Sep 17 13:56:00.111583 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.111557    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 13:56:00.112012 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.111975    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw","openshift-cluster-node-tuning-operator/tuned-lrnzs","openshift-image-registry/node-ca-dbsps","openshift-multus/multus-4mzr9","openshift-multus/network-metrics-daemon-c4v66","openshift-network-diagnostics/network-check-target-9lmwq","openshift-network-operator/iptables-alerter-kwxl2","openshift-ovn-kubernetes/ovnkube-node-6lq2d","kube-system/global-pull-secret-syncer-vpfvq","kube-system/konnectivity-agent-zjt8f","kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal","openshift-dns/node-resolver-sw8gq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal","openshift-multus/multus-additional-cni-plugins-qtfw6"]
Sep 17 13:56:00.114650 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.114626    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.115062 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.114981    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 13:56:00.117085 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.117062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.117617 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.117598    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.117861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.117844    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.118057 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.118042    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 13:56:00.118130 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.118045    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7jffz\""
Sep 17 13:56:00.119431 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.119322    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.120003 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.119984    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 13:56:00.120094 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120032    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 13:56:00.120235 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120220    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.120293 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120283    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 13:56:00.120987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120925    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.120987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120925    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rvw22\""
Sep 17 13:56:00.120987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.120966    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 13:56:00.121491 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.121476    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.122161 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.121653    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.122161 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.121618    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 13:56:00.122161 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.121705    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.122161 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.122066    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4p85j\""
Sep 17 13:56:00.124137 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124103    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 13:56:00.124499 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124249    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.124499 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124253    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.124499 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124362    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 13:56:00.124499 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124461    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.124708 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.124524    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:00.124708 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124632    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cgk4l\""
Sep 17 13:56:00.124819 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124757    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-netns\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124819 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-ovn\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124819 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124797    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-env-overrides\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124819 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124811    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-224m5\" (UniqueName: \"kubernetes.io/projected/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-kube-api-access-224m5\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124952 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124826    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28214052-caa9-40be-86a9-ca5ae2dbd094-host-slash\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.124952 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124863    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8nrt\" (UniqueName: \"kubernetes.io/projected/28214052-caa9-40be-86a9-ca5ae2dbd094-kube-api-access-k8nrt\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.124952 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-kubelet\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124952 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124919    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-systemd-units\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.124952 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124941    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-slash\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.124959    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-var-lib-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125000    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-etc-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125017    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125035    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-systemd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125049    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-node-log\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125078    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-log-socket\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125092    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-bin\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125107    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125142 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125135    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-config\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125459 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125173    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-script-lib\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125459 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125233    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125459 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125259    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-netd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125459 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125279    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovn-node-metrics-cert\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.125459 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.125318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28214052-caa9-40be-86a9-ca5ae2dbd094-iptables-alerter-script\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.126449 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.126429    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 13:56:00.126609 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.126593    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:00.126687 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.126658    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:00.129045 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.128720    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.129045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.128819    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:00.131425 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.131407    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.133887 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.133865    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.134537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.134517    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h5nkd\""
Sep 17 13:56:00.135557 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.134840    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 13:56:00.135557 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.134916    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.135557 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.135049    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.136254 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.136236    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.136423 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.136406    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.136908 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.136889    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8kz7q\""
Sep 17 13:56:00.137001 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.136954    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.139030 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.139008    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wkfzt\""
Sep 17 13:56:00.139119 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.139042    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 13:56:00.139433 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.139418    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 13:56:00.141954 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.141935    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.144498 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.144479    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.144498 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.144490    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 13:56:00.144498 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.144499    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5p4g4\""
Sep 17 13:56:00.144722 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.144698    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 13:56:00.147107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.147086    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 13:56:00.147196 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.147119    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2qxlc\""
Sep 17 13:56:00.147196 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.147122    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 13:56:00.150282 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.150263    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mnfdg"
Sep 17 13:56:00.156860 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.156843    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mnfdg"
Sep 17 13:56:00.181302 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.181276    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"352a6f4db44839b0aac72f56039c3b11","Type":"ContainerStarted","Data":"a6be05c2d4bb29f6996d97df0b96c65150a54e8eb28d2824c158dee33a84a8fb"}
Sep 17 13:56:00.197651 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.197619    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 13:56:00.217167 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.217119    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 13:56:00.226054 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226033    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-modprobe-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226122 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226066    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-hostroot\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226122 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-kubelet-config\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.226122 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226102    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.226122 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226117    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-multus\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226136    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-var-lib-kubelet\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226169    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-host\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-serviceca\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226218    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-netns\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cnibin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28214052-caa9-40be-86a9-ca5ae2dbd094-iptables-alerter-script\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.226309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226304    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-lib-modules\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226606 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226407    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-tuned\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226606 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226451    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlzpx\" (UniqueName: \"kubernetes.io/projected/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-kube-api-access-mlzpx\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.226606 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226484    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-system-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226606 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226509    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-socket-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.226606 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226537    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-kubelet\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226604    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-kubelet\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226600    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-var-lib-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-etc-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226648    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-var-lib-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-kubernetes\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226694    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-system-cni-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226699    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-etc-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226716    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226760    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-bin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-config\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226807    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwrtt\" (UniqueName: \"kubernetes.io/projected/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-kube-api-access-dwrtt\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226814    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/28214052-caa9-40be-86a9-ca5ae2dbd094-iptables-alerter-script\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cni-binary-copy\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.226861 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-socket-dir-parent\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-multus-certs\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226896    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.226966    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227002    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-netd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227025    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-conf\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227042    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227048    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-run\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227105    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-netd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227103    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227135    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-netns\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-sys-fs\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227258    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28214052-caa9-40be-86a9-ca5ae2dbd094-host-slash\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227285    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-run-netns\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227283    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-systemd-units\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.227579 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227315    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-config\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227322    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-systemd-units\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-bin\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227360    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227366    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/28214052-caa9-40be-86a9-ca5ae2dbd094-host-slash\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227369    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-openvswitch\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227380    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cnibin\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227392    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-cni-bin\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227403    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-kubelet\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227421    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227411    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227436    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ac6fa6-352e-4052-8d50-dfc868b889f0-tmp-dir\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-dbus\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227480    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8qf\" (UniqueName: \"kubernetes.io/projected/10ac6fa6-352e-4052-8d50-dfc868b889f0-kube-api-access-9v8qf\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227497    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-host\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227511    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-etc-kubernetes\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.228332 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-ovn\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227542    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-224m5\" (UniqueName: \"kubernetes.io/projected/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-kube-api-access-224m5\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227564    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-device-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227577    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-ovn\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227587    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvqg8\" (UniqueName: \"kubernetes.io/projected/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kube-api-access-pvqg8\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227610    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdb2c331-e40e-42dd-963c-5b2230d90e5a-konnectivity-ca\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-node-log\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-systemd\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-tmp\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227693    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-node-log\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-k8s-cni-cncf-io\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-registration-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227766    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-systemd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227845    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-script-lib\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227864    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-run-systemd\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227871    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229040 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227901    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwqv\" (UniqueName: \"kubernetes.io/projected/91d1df0a-327f-4288-970d-cc1c0ce5c884-kube-api-access-fkwqv\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227926    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227950    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-conf-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.227973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228000    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovn-node-metrics-cert\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228026    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysconfig\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-env-overrides\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228168    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-sys\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228193    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/10ac6fa6-352e-4052-8d50-dfc868b889f0-hosts-file\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228224    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-daemon-config\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228240    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovnkube-script-lib\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqjwh\" (UniqueName: \"kubernetes.io/projected/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-kube-api-access-vqjwh\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228278    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdb2c331-e40e-42dd-963c-5b2230d90e5a-agent-certs\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228304    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwr65\" (UniqueName: \"kubernetes.io/projected/b93295b6-74b9-4536-a67f-532b4c8f49b8-kube-api-access-hwr65\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8nrt\" (UniqueName: \"kubernetes.io/projected/28214052-caa9-40be-86a9-ca5ae2dbd094-kube-api-access-k8nrt\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228407    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-env-overrides\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-slash\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.229658 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-log-socket\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.230311 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-host-slash\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.230311 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228586    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-log-socket\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.230311 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.228697    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 13:56:00.232665 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.232506    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-ovn-node-metrics-cert\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.237664 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.237646    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-224m5\" (UniqueName: \"kubernetes.io/projected/dc31ad42-f268-4415-9afb-7ebe9d38c9b4-kube-api-access-224m5\") pod \"ovnkube-node-6lq2d\" (UID: \"dc31ad42-f268-4415-9afb-7ebe9d38c9b4\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.237766 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.237747    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8nrt\" (UniqueName: \"kubernetes.io/projected/28214052-caa9-40be-86a9-ca5ae2dbd094-kube-api-access-k8nrt\") pod \"iptables-alerter-kwxl2\" (UID: \"28214052-caa9-40be-86a9-ca5ae2dbd094\") " pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.329895    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.329955    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ac6fa6-352e-4052-8d50-dfc868b889f0-tmp-dir\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.329980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-dbus\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330009    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8qf\" (UniqueName: \"kubernetes.io/projected/10ac6fa6-352e-4052-8d50-dfc868b889f0-kube-api-access-9v8qf\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330048    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-host\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-etc-kubernetes\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330112    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-device-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvqg8\" (UniqueName: \"kubernetes.io/projected/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kube-api-access-pvqg8\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330163    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdb2c331-e40e-42dd-963c-5b2230d90e5a-konnectivity-ca\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330192    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-systemd\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-tmp\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330262    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-k8s-cni-cncf-io\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-registration-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330342    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330370    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330409    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwqv\" (UniqueName: \"kubernetes.io/projected/91d1df0a-327f-4288-970d-cc1c0ce5c884-kube-api-access-fkwqv\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.330656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330441    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330503    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-conf-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330540    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysconfig\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330779    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-sys\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330852    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330858    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-sys\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330898    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/10ac6fa6-352e-4052-8d50-dfc868b889f0-hosts-file\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330962    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10ac6fa6-352e-4052-8d50-dfc868b889f0-tmp-dir\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331028    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-daemon-config\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqjwh\" (UniqueName: \"kubernetes.io/projected/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-kube-api-access-vqjwh\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331076    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysconfig\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.331081    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.330930    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-k8s-cni-cncf-io\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331126    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdb2c331-e40e-42dd-963c-5b2230d90e5a-agent-certs\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.331214    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:00.831159989 +0000 UTC m=+3.068037464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331210    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/10ac6fa6-352e-4052-8d50-dfc868b889f0-hosts-file\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.331571 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331121    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-registration-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331230    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-systemd\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331290    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bdb2c331-e40e-42dd-963c-5b2230d90e5a-konnectivity-ca\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-device-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331388    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-etc-kubernetes\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331492    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-dbus\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331670    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331748    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-host\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331845    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-binary-copy\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.331864    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331884    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-daemon-config\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwr65\" (UniqueName: \"kubernetes.io/projected/b93295b6-74b9-4536-a67f-532b4c8f49b8-kube-api-access-hwr65\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.331930    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:00.831914341 +0000 UTC m=+3.068791799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331957    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-conf-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.331985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-modprobe-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332018    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-hostroot\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332044    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-kubelet-config\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.332339 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332109    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-multus\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332136    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-modprobe-d\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332139    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-var-lib-kubelet\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332184    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46b9c53f-aba0-48fc-b364-d9c3a167f498-kubelet-config\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332186    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-host\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332228    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-multus\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332249    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-hostroot\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332281    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-host\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332293    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-serviceca\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332187    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-var-lib-kubelet\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332324    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-netns\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332373    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-netns\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332405    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cnibin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-lib-modules\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332485    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-tuned\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332517    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cnibin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlzpx\" (UniqueName: \"kubernetes.io/projected/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-kube-api-access-mlzpx\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.333143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-system-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-socket-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332647    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-lib-modules\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-kubernetes\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332743    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-system-cni-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332802    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-serviceca\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332828    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332856    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332870    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-system-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332907    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-bin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332924    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-kubernetes\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332938    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwrtt\" (UniqueName: \"kubernetes.io/projected/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-kube-api-access-dwrtt\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.332950    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-system-cni-dir\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333018    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-cni-dir\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333029    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-socket-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333066    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cni-binary-copy\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333081    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-cni-bin\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-socket-dir-parent\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.334919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-multus-certs\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333248    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-run-multus-certs\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-conf\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333349    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333353    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-run\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333385    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333417    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-sys-fs\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333504    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cnibin\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333534    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-sysctl-conf\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333546    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-sys-fs\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333554    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-kubelet\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333593    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-cnibin\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333630    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-host-var-lib-kubelet\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333644    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91d1df0a-327f-4288-970d-cc1c0ce5c884-run\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.335406 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.333755    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334196    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-tmp\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334199    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-cni-binary-copy\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334195    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-multus-socket-dir-parent\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334380    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334416    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bdb2c331-e40e-42dd-963c-5b2230d90e5a-agent-certs\") pod \"konnectivity-agent-zjt8f\" (UID: \"bdb2c331-e40e-42dd-963c-5b2230d90e5a\") " pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.335907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.334818    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91d1df0a-327f-4288-970d-cc1c0ce5c884-etc-tuned\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.342875 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.342853    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwqv\" (UniqueName: \"kubernetes.io/projected/91d1df0a-327f-4288-970d-cc1c0ce5c884-kube-api-access-fkwqv\") pod \"tuned-lrnzs\" (UID: \"91d1df0a-327f-4288-970d-cc1c0ce5c884\") " pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.343053 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.343036    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8qf\" (UniqueName: \"kubernetes.io/projected/10ac6fa6-352e-4052-8d50-dfc868b889f0-kube-api-access-9v8qf\") pod \"node-resolver-sw8gq\" (UID: \"10ac6fa6-352e-4052-8d50-dfc868b889f0\") " pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.343374 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.343353    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqjwh\" (UniqueName: \"kubernetes.io/projected/1bb4ff94-8ceb-42d0-b4e3-2b30943fb349-kube-api-access-vqjwh\") pod \"multus-4mzr9\" (UID: \"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349\") " pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.343424 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.343411    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwr65\" (UniqueName: \"kubernetes.io/projected/b93295b6-74b9-4536-a67f-532b4c8f49b8-kube-api-access-hwr65\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.343989 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.343971    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvqg8\" (UniqueName: \"kubernetes.io/projected/cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56-kube-api-access-pvqg8\") pod \"aws-ebs-csi-driver-node-6jgbw\" (UID: \"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.346066 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.346047    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:00.346162 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.346072    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:00.346162 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.346085    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:00.346162 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.346135    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:00.846120495 +0000 UTC m=+3.082997950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:00.348196 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.348176    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlzpx\" (UniqueName: \"kubernetes.io/projected/eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69-kube-api-access-mlzpx\") pod \"multus-additional-cni-plugins-qtfw6\" (UID: \"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69\") " pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.348536 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.348519    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwrtt\" (UniqueName: \"kubernetes.io/projected/ddc3a6a1-6e86-47ca-811a-4334fbcffbbb-kube-api-access-dwrtt\") pod \"node-ca-dbsps\" (UID: \"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb\") " pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.427543 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.427515    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kwxl2"
Sep 17 13:56:00.436224 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.436206    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:00.444940 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.444921    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dbsps"
Sep 17 13:56:00.452778 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.452759    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4mzr9"
Sep 17 13:56:00.460218 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.460198    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw"
Sep 17 13:56:00.467857 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.467813    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lrnzs"
Sep 17 13:56:00.474309 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.474295    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:00.482868 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.482850    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qtfw6"
Sep 17 13:56:00.487428 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.487412    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sw8gq"
Sep 17 13:56:00.836603 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.836512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:00.836603 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.836548    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:00.836859 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.836688    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:00.836859 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.836709    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:00.836859 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.836775    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:01.836752721 +0000 UTC m=+4.073630476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:00.836859 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.836797    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:01.836790058 +0000 UTC m=+4.073667512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:00.937838 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:00.937803    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:00.938023 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.937997    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:00.938023 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.938027    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:00.938168 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.938038    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:00.938168 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:00.938095    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:01.938074545 +0000 UTC m=+4.174952002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:01.029263 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:56:01.029239    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc31ad42_f268_4415_9afb_7ebe9d38c9b4.slice/crio-f970ed51d10e90215325e3fa54d8318425100e1dd5e62e204fc8e41fe5682cc5 WatchSource:0}: Error finding container f970ed51d10e90215325e3fa54d8318425100e1dd5e62e204fc8e41fe5682cc5: Status 404 returned error can't find the container with id f970ed51d10e90215325e3fa54d8318425100e1dd5e62e204fc8e41fe5682cc5
Sep 17 13:56:01.030593 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:56:01.030571    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb4ff94_8ceb_42d0_b4e3_2b30943fb349.slice/crio-bc17ce1d1afa2d7ee635e2980921ac02b2fb74f28d9259e5a562149afa7c39a0 WatchSource:0}: Error finding container bc17ce1d1afa2d7ee635e2980921ac02b2fb74f28d9259e5a562149afa7c39a0: Status 404 returned error can't find the container with id bc17ce1d1afa2d7ee635e2980921ac02b2fb74f28d9259e5a562149afa7c39a0
Sep 17 13:56:01.031535 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:56:01.031479    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10ac6fa6_352e_4052_8d50_dfc868b889f0.slice/crio-54fed94c1a3839b17314be8975ebd30612bd74b3c54e524348fbba4423e705bd WatchSource:0}: Error finding container 54fed94c1a3839b17314be8975ebd30612bd74b3c54e524348fbba4423e705bd: Status 404 returned error can't find the container with id 54fed94c1a3839b17314be8975ebd30612bd74b3c54e524348fbba4423e705bd
Sep 17 13:56:01.034495 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:56:01.034416    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc3a6a1_6e86_47ca_811a_4334fbcffbbb.slice/crio-e772e428adfb9e2637a01d01dfa4f6c936704c0272359dd083fc6d043975a527 WatchSource:0}: Error finding container e772e428adfb9e2637a01d01dfa4f6c936704c0272359dd083fc6d043975a527: Status 404 returned error can't find the container with id e772e428adfb9e2637a01d01dfa4f6c936704c0272359dd083fc6d043975a527
Sep 17 13:56:01.160288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.160253    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 13:51:00 +0000 UTC" deadline="2028-03-06 22:18:07.070548571 +0000 UTC"
Sep 17 13:56:01.160288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.160280    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12872h22m5.910271749s"
Sep 17 13:56:01.184004 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.183970    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwxl2" event={"ID":"28214052-caa9-40be-86a9-ca5ae2dbd094","Type":"ContainerStarted","Data":"393808922c52919d1083449accb54067301489c8ab1dabbefa0e899f788b17e4"}
Sep 17 13:56:01.185150 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.185128    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjt8f" event={"ID":"bdb2c331-e40e-42dd-963c-5b2230d90e5a","Type":"ContainerStarted","Data":"a1fc6094aa4454d7b90c1bccb6a2e4d04e65a229684dc4089a622204bbf32aaf"}
Sep 17 13:56:01.186162 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.186142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw" event={"ID":"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56","Type":"ContainerStarted","Data":"1003a2c968b795138f6d9cc53da994799d84cb745c49bb22bb178cb06f47b2d6"}
Sep 17 13:56:01.187285 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.187261    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dbsps" event={"ID":"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb","Type":"ContainerStarted","Data":"e772e428adfb9e2637a01d01dfa4f6c936704c0272359dd083fc6d043975a527"}
Sep 17 13:56:01.188441 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.188418    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"f970ed51d10e90215325e3fa54d8318425100e1dd5e62e204fc8e41fe5682cc5"}
Sep 17 13:56:01.190512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.190483    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" event={"ID":"c4f272aba07b562b95d8ca5dc3318b8a","Type":"ContainerStarted","Data":"15e2a94217a9a9c1d69b345e8028ea577346a7263fc98fc6b94176c8efc092c8"}
Sep 17 13:56:01.191599 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.191582    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerStarted","Data":"fa510e229b8532120e33e3a2db296a13195f98ffa1c3a1342bd2261a4225d1cc"}
Sep 17 13:56:01.192669 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.192649    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lrnzs" event={"ID":"91d1df0a-327f-4288-970d-cc1c0ce5c884","Type":"ContainerStarted","Data":"9f561c2892b810d8835e5efdd69e9ce19279c72cbfe5cf4788e24dc5e58b5d13"}
Sep 17 13:56:01.193748 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.193712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sw8gq" event={"ID":"10ac6fa6-352e-4052-8d50-dfc868b889f0","Type":"ContainerStarted","Data":"54fed94c1a3839b17314be8975ebd30612bd74b3c54e524348fbba4423e705bd"}
Sep 17 13:56:01.194967 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.194949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mzr9" event={"ID":"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349","Type":"ContainerStarted","Data":"bc17ce1d1afa2d7ee635e2980921ac02b2fb74f28d9259e5a562149afa7c39a0"}
Sep 17 13:56:01.206799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.206755    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" podStartSLOduration=2.206722208 podStartE2EDuration="2.206722208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:01.206403822 +0000 UTC m=+3.443281300" watchObservedRunningTime="2026-09-17 13:56:01.206722208 +0000 UTC m=+3.443599721"
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.844844    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.844896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.845013    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.845072    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:03.845054893 +0000 UTC m=+6.081932365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.845761    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:01.845849 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.845815    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:03.845798783 +0000 UTC m=+6.082676242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:01.948310 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:01.945697    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:01.948310 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.945936    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:01.948310 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.945956    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:01.948310 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.945969    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:01.948310 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:01.946024    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:03.946007474 +0000 UTC m=+6.182884932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:02.161243 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.161181    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 13:51:00 +0000 UTC" deadline="2028-03-18 17:58:29.027428026 +0000 UTC"
Sep 17 13:56:02.161243 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.161241    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13156h2m26.866190552s"
Sep 17 13:56:02.178282 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.177946    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:02.178282 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:02.178066    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:02.178486 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.178441    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:02.178626 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:02.178545    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:02.178626 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.178608    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:02.178759 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:02.178678    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:02.215510 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.215437    2570 generic.go:356] "Generic (PLEG): container finished" podID="352a6f4db44839b0aac72f56039c3b11" containerID="33c9d2aa97cde18d248a6391f55c491fc4781fc9fbb5d9ce7114cd104f23e3c5" exitCode=0
Sep 17 13:56:02.215912 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:02.215848    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"352a6f4db44839b0aac72f56039c3b11","Type":"ContainerDied","Data":"33c9d2aa97cde18d248a6391f55c491fc4781fc9fbb5d9ce7114cd104f23e3c5"}
Sep 17 13:56:03.234167 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:03.234107    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"352a6f4db44839b0aac72f56039c3b11","Type":"ContainerStarted","Data":"2580187dd9e7ee442792783725228a8e2d48d3b90002bf3668ac44f0ee9434ed"}
Sep 17 13:56:03.865697 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:03.865656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:03.865941 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:03.865705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:03.865941 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.865852    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:03.865941 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.865913    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:07.865893806 +0000 UTC m=+10.102771263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:03.866315 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.866293    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:03.866398 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.866351    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:07.866333866 +0000 UTC m=+10.103211324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:03.966300 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:03.966264    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:03.966482 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.966471    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:03.966545 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.966492    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:03.966545 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.966505    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:03.966642 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:03.966563    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:07.966546082 +0000 UTC m=+10.203423541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:04.177253    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:04.177377    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:04.177755    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:04.177860    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:04.177904    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:04.178079 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:04.177981    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:06.177859 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:06.177829    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:06.178290 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:06.177954    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:06.178348 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:06.178330    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:06.178446 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:06.178426    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:06.178498 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:06.178484    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:06.178563 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:06.178546    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:07.905787 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:07.905274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:07.905787 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:07.905541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:07.905787 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:07.905559    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:07.905787 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:07.905633    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:15.905612361 +0000 UTC m=+18.142489838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:07.905787 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:07.905693    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:07.906436 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:07.906414    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:15.906390326 +0000 UTC m=+18.143267798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:08.006699 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:08.006659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:08.006904 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.006826    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:08.006904 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.006847    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:08.006904 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.006860    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:08.007061 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.006916    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:16.006897436 +0000 UTC m=+18.243774892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:08.176440 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:08.176370    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:08.176585 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.176451    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:08.176785 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:08.176761    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:08.176926 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.176857    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:08.177335 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:08.177314    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:08.177435 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:08.177415    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:10.175788 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:10.175751    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:10.176260 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:10.175750    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:10.176260 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:10.175901    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:10.176260 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:10.175963    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:10.176260 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:10.175751    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:10.176260 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:10.176037    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:12.175572 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:12.175540    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:12.175572 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:12.175567    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:12.176067 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:12.175547    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:12.176067 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:12.175653    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:12.176067 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:12.175746    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:12.176067 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:12.175814    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:14.175684 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:14.175646    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:14.176188 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:14.175647    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:14.176188 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:14.175782    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:14.176188 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:14.175661    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:14.176188 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:14.175861    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:14.176188 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:14.175925    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:15.969880    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:15.969934    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:15.970059    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:15.970061    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:15.970118    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:31.970101086 +0000 UTC m=+34.206978542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:15.970430 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:15.970148    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:31.97012853 +0000 UTC m=+34.207005985 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:16.071224 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:16.071192    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:16.071396 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.071377    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:16.071449 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.071406    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:16.071449 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.071420    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:16.071524 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.071471    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:56:32.071458124 +0000 UTC m=+34.308335583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:16.175250 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:16.175219    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:16.175250 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:16.175232    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:16.175450 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.175342    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:16.178512 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.175832    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:16.178512 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:16.176497    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:16.178699 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:16.178672    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:18.176053 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:18.176020    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:18.176525 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:18.176117    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:18.176525 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:18.176205    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:18.176525 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:18.176317    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:18.176525 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:18.176372    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:18.176525 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:18.176442    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:20.175526 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.175266    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:20.175526 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.175277    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:20.176458 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.175510    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:20.176458 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:20.175795    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:20.176458 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:20.175942    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:20.176458 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:20.176065    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:20.267378 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.267345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lrnzs" event={"ID":"91d1df0a-327f-4288-970d-cc1c0ce5c884","Type":"ContainerStarted","Data":"d8d737d6ac2c045b03c2994f45df4d1f486bc5e8f57bf47cd25fc3ea2285921b"}
Sep 17 13:56:20.269047 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.269014    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sw8gq" event={"ID":"10ac6fa6-352e-4052-8d50-dfc868b889f0","Type":"ContainerStarted","Data":"cbda417028e5b3833ae396b045e719eb70007429e95b5414703d617190bf86ac"}
Sep 17 13:56:20.270816 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.270788    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4mzr9" event={"ID":"1bb4ff94-8ceb-42d0-b4e3-2b30943fb349","Type":"ContainerStarted","Data":"f66aa9cf180f2bac83d08d210603df0938bcb0b476ec193916dba01d4cbb0b53"}
Sep 17 13:56:20.272428 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.272398    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zjt8f" event={"ID":"bdb2c331-e40e-42dd-963c-5b2230d90e5a","Type":"ContainerStarted","Data":"961eb41d387cc649a02d1b7ea2b40ee8bf123ce432cc5a747e0e62f1fcf5bcf2"}
Sep 17 13:56:20.274176 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.274155    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw" event={"ID":"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56","Type":"ContainerStarted","Data":"bfb4678fb62f5a0d346516223f598f3a17a8f0dcfb991233cea12f65795443d9"}
Sep 17 13:56:20.275777 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.275719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dbsps" event={"ID":"ddc3a6a1-6e86-47ca-811a-4334fbcffbbb","Type":"ContainerStarted","Data":"08d0903c7a8f541fe246c11f8821d441880d2b7e81b2651324bfd180f535b505"}
Sep 17 13:56:20.279167 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279144    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"b455df7853842abed02309021fbdd1ccdac12a17684022171ee1d8e158870991"}
Sep 17 13:56:20.279257 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279211    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"c696720549765cfcdcbecfc783e93563700f3f568d319e97e277dc0064819004"}
Sep 17 13:56:20.279257 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279254    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"43c737a6d8fa3606148e165cac5477a228ff7cf6f540c2c22ffc5c3b81f2393f"}
Sep 17 13:56:20.279362 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279303    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"147cf2978a1c2c4edaece51a6b0a272bc900e3ce14ba3a76bba0d309243a39c1"}
Sep 17 13:56:20.279362 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279348    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"5c543264fae1e09879a9d158d471ffc0124662323a5e88ee128660914dafb961"}
Sep 17 13:56:20.279487 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.279376    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"1df90145907ab131b1d25fca6095f2109d245cecb7f5819c4e57969c066ba666"}
Sep 17 13:56:20.280882 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.280856    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="d40f79eca15056bebdffe9ff083e5ed40bb798a50aefaae3c0fc1c563f5218c8" exitCode=0
Sep 17 13:56:20.280882 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.280884    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"d40f79eca15056bebdffe9ff083e5ed40bb798a50aefaae3c0fc1c563f5218c8"}
Sep 17 13:56:20.304498 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.304461    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" podStartSLOduration=21.304448457 podStartE2EDuration="21.304448457s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:03.25333806 +0000 UTC m=+5.490215537" watchObservedRunningTime="2026-09-17 13:56:20.304448457 +0000 UTC m=+22.541325933"
Sep 17 13:56:20.340814 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.340778    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lrnzs" podStartSLOduration=4.1858095 podStartE2EDuration="22.340764615s" totalImagesPullingTime="18.154955115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:20.304803737 +0000 UTC m=+22.541681215" watchObservedRunningTime="2026-09-17 13:56:20.340764615 +0000 UTC m=+22.577642092"
Sep 17 13:56:20.389987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.389951    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zjt8f" podStartSLOduration=4.311524083 podStartE2EDuration="22.389939694s" totalImagesPullingTime="18.078415611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:20.340714531 +0000 UTC m=+22.577592008" watchObservedRunningTime="2026-09-17 13:56:20.389939694 +0000 UTC m=+22.626817171"
Sep 17 13:56:20.431393 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.431336    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sw8gq" podStartSLOduration=4.270651445 podStartE2EDuration="22.431325282s" totalImagesPullingTime="18.160673837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:20.431101823 +0000 UTC m=+22.667979310" watchObservedRunningTime="2026-09-17 13:56:20.431325282 +0000 UTC m=+22.668202759"
Sep 17 13:56:20.475859 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.475818    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dbsps" podStartSLOduration=4.317574981 podStartE2EDuration="22.475806365s" totalImagesPullingTime="18.158231384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:20.453001084 +0000 UTC m=+22.689878561" watchObservedRunningTime="2026-09-17 13:56:20.475806365 +0000 UTC m=+22.712683841"
Sep 17 13:56:20.475950 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.475894    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4mzr9" podStartSLOduration=4.128010251 podStartE2EDuration="22.475885681s" totalImagesPullingTime="18.34787543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:20.475868663 +0000 UTC m=+22.712746141" watchObservedRunningTime="2026-09-17 13:56:20.475885681 +0000 UTC m=+22.712763159"
Sep 17 13:56:20.772699 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:20.772556    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 13:56:21.186972 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.186870    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T13:56:20.772696304Z","UUID":"b9a281d4-f33b-4de8-94b3-5b83d855fb33","Handler":null,"Name":"","Endpoint":""}
Sep 17 13:56:21.189293 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.189270    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 13:56:21.189424 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.189306    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 13:56:21.285311 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.285273    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kwxl2" event={"ID":"28214052-caa9-40be-86a9-ca5ae2dbd094","Type":"ContainerStarted","Data":"3932fc29e43951dc25d1d58132baf69b51f33f7464bffd5242c92849cf893f7b"}
Sep 17 13:56:21.287348 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.287319    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw" event={"ID":"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56","Type":"ContainerStarted","Data":"a8972f74e1ab98c99e8843868c7423473701ec0effebdeb9baa7b6712b696459"}
Sep 17 13:56:21.302673 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:21.302635    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kwxl2" podStartSLOduration=5.138888519 podStartE2EDuration="23.302623078s" totalImagesPullingTime="18.163734559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:21.302498277 +0000 UTC m=+23.539375754" watchObservedRunningTime="2026-09-17 13:56:21.302623078 +0000 UTC m=+23.539500555"
Sep 17 13:56:22.175829 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.175801    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:22.175829 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.175817    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:22.176055 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:22.175914    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:22.176055 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.175994    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:22.176055 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:22.175994    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:22.176201 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:22.176073    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:22.291830 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.291754    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw" event={"ID":"cb3973ac-4e07-4a7f-bbb8-1a3759ed0c56","Type":"ContainerStarted","Data":"afaa539b3fb390ec509fd14608f7277a8df542e4ef1ccad22de1692c2b6970ba"}
Sep 17 13:56:22.295894 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.295861    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"fd6416c49af9ca5baeaee052db32c101c39932c0cf49e5f9edc283d8d3df45e0"}
Sep 17 13:56:22.316999 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:22.316961    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jgbw" podStartSLOduration=4.012030703 podStartE2EDuration="24.316940701s" totalImagesPullingTime="20.304909998s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:22.316791551 +0000 UTC m=+24.553669030" watchObservedRunningTime="2026-09-17 13:56:22.316940701 +0000 UTC m=+24.553818177"
Sep 17 13:56:23.533455 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:23.533419    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:23.534223 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:23.534080    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:24.178976 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:24.178945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:24.179124 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:24.179019    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:24.179124 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:24.179120    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:24.179208 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:24.179130    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:24.179260 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:24.179213    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:24.179312 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:24.179279    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:24.300744 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:24.300695    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:24.301230 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:24.301206    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zjt8f"
Sep 17 13:56:26.175114 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.174955    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:26.175561 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.174973    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:26.175561 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.174974    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:26.175561 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:26.175286    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:26.175561 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:26.175187    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:26.175561 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:26.175408    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:26.307745 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.307697    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" event={"ID":"dc31ad42-f268-4415-9afb-7ebe9d38c9b4","Type":"ContainerStarted","Data":"1fd29fc267431907d41a258a586e1956013f58f211976ff710c6028eb58b917b"}
Sep 17 13:56:26.307969 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.307946    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:26.309818 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.309797    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="676c67783a8ea1e7eddc0cfa8cdc79cfae0800c82fb516741e81128393e7130d" exitCode=0
Sep 17 13:56:26.309933 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.309876    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"676c67783a8ea1e7eddc0cfa8cdc79cfae0800c82fb516741e81128393e7130d"}
Sep 17 13:56:26.323342 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.323323    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:26.423644 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:26.423608    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d" podStartSLOduration=10.057490294 podStartE2EDuration="28.423595155s" totalImagesPullingTime="18.366104861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:26.37229427 +0000 UTC m=+28.609171747" watchObservedRunningTime="2026-09-17 13:56:26.423595155 +0000 UTC m=+28.660472633"
Sep 17 13:56:27.313692 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.313656    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="74d70c68cbf5ade9fe7ca368330257934a1cc291c2965b242705548a4ff75113" exitCode=0
Sep 17 13:56:27.314130 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.313749    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"74d70c68cbf5ade9fe7ca368330257934a1cc291c2965b242705548a4ff75113"}
Sep 17 13:56:27.314384 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.314360    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:27.314538 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.314440    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:27.330585 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.330470    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:56:27.504789 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.504765    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9lmwq"]
Sep 17 13:56:27.504925 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.504886    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:27.504983 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:27.504968    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:27.505420 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.505398    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpfvq"]
Sep 17 13:56:27.505504 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.505494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:27.505608 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:27.505586    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:27.507500 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.507478    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c4v66"]
Sep 17 13:56:27.507597 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:27.507576    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:27.507692 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:27.507673    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:28.319104 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:28.319070    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="201e1fb7e1dbd55a97059081df49b6b17465c84a757b0b91ccf337f5493cb080" exitCode=0
Sep 17 13:56:28.319514 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:28.319161    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"201e1fb7e1dbd55a97059081df49b6b17465c84a757b0b91ccf337f5493cb080"}
Sep 17 13:56:29.175308 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:29.175278    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:29.175464 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:29.175347    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:29.175464 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:29.175449    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:29.175571 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:29.175455    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:29.175571 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:29.175540    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:29.175657 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:29.175593    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:31.175964 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:31.175917    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:31.175964 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:31.175962    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:31.176496 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:31.175963    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:31.176496 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.176040    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9lmwq" podUID="021db228-863a-4003-8cfc-218ec61e30c5"
Sep 17 13:56:31.176496 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.176145    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c4v66" podUID="b93295b6-74b9-4536-a67f-532b4c8f49b8"
Sep 17 13:56:31.176496 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.176235    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vpfvq" podUID="46b9c53f-aba0-48fc-b364-d9c3a167f498"
Sep 17 13:56:31.984080 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:31.984051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:31.984265 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:31.984093    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:31.984265 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.984236    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:31.984265 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.984242    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:31.984404 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.984305    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:03.984290216 +0000 UTC m=+66.221167675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 13:56:31.984404 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:31.984320    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret podName:46b9c53f-aba0-48fc-b364-d9c3a167f498 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:03.984313387 +0000 UTC m=+66.221190843 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret") pod "global-pull-secret-syncer-vpfvq" (UID: "46b9c53f-aba0-48fc-b364-d9c3a167f498") : object "kube-system"/"original-pull-secret" not registered
Sep 17 13:56:32.085440 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:32.085412    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:32.085607 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:32.085588    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 13:56:32.085674 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:32.085616    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 13:56:32.085674 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:32.085630    2570 projected.go:196] Error preparing data for projected volume kube-api-access-4jf6f for pod openshift-network-diagnostics/network-check-target-9lmwq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:32.085787 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:32.085695    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f podName:021db228-863a-4003-8cfc-218ec61e30c5 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:04.085667813 +0000 UTC m=+66.322545274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4jf6f" (UniqueName: "kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f") pod "network-check-target-9lmwq" (UID: "021db228-863a-4003-8cfc-218ec61e30c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 13:56:33.064988 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.064964    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeReady"
Sep 17 13:56:33.065377 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.065116    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 13:56:33.117025 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.116998    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zrswq"]
Sep 17 13:56:33.130594 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.130574    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2j5sd"]
Sep 17 13:56:33.130749 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.130708    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.139310 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.139272    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 13:56:33.139659 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.139575    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 13:56:33.139659 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.139594    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 13:56:33.139891 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.139873    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wntfx\""
Sep 17 13:56:33.144533 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.144516    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrswq"]
Sep 17 13:56:33.144620 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.144544    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2j5sd"]
Sep 17 13:56:33.144670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.144622    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.150434 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.150416    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f8wrh\""
Sep 17 13:56:33.150523 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.150449    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 13:56:33.150523 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.150452    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 13:56:33.174974 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.174957    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:56:33.175079 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.174983    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:56:33.175155 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.175143    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:56:33.178842 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.178826    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gs47f\""
Sep 17 13:56:33.178944 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.178874    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cpdmf\""
Sep 17 13:56:33.179091 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.179078    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 13:56:33.179377 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.179362    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 13:56:33.179913 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.179890    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 13:56:33.179913 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.179892    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 13:56:33.295551 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295528    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzb2\" (UniqueName: \"kubernetes.io/projected/9e4abac5-542b-4c97-9875-df2a128c7d61-kube-api-access-nfzb2\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.295661 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295567    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c05b88d-95b9-4735-bfd6-f896023da4be-tmp-dir\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.295661 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295620    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.295661 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295641    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq774\" (UniqueName: \"kubernetes.io/projected/9c05b88d-95b9-4735-bfd6-f896023da4be-kube-api-access-wq774\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.295796 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4abac5-542b-4c97-9875-df2a128c7d61-cert\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.295796 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.295761    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c05b88d-95b9-4735-bfd6-f896023da4be-config-volume\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.332979 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.332957    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="8cb463f312db14849deebf97f8a3fece46acb5df894ce2b9a67555ab7663264b" exitCode=0
Sep 17 13:56:33.333064 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.332990    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"8cb463f312db14849deebf97f8a3fece46acb5df894ce2b9a67555ab7663264b"}
Sep 17 13:56:33.396186 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4abac5-542b-4c97-9875-df2a128c7d61-cert\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.396300 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396214    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c05b88d-95b9-4735-bfd6-f896023da4be-config-volume\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.396300 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396248    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzb2\" (UniqueName: \"kubernetes.io/projected/9e4abac5-542b-4c97-9875-df2a128c7d61-kube-api-access-nfzb2\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.396300 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396273    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c05b88d-95b9-4735-bfd6-f896023da4be-tmp-dir\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.396452 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396300    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.396452 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396325    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq774\" (UniqueName: \"kubernetes.io/projected/9c05b88d-95b9-4735-bfd6-f896023da4be-kube-api-access-wq774\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.396679 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:33.396654    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:33.396867 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396659    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c05b88d-95b9-4735-bfd6-f896023da4be-tmp-dir\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.396867 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:33.396758    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:56:33.896718834 +0000 UTC m=+36.133596305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:33.396989 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.396918    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c05b88d-95b9-4735-bfd6-f896023da4be-config-volume\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.401383 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.401260    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e4abac5-542b-4c97-9875-df2a128c7d61-cert\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.414713 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.414693    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq774\" (UniqueName: \"kubernetes.io/projected/9c05b88d-95b9-4735-bfd6-f896023da4be-kube-api-access-wq774\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.417140 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.417124    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzb2\" (UniqueName: \"kubernetes.io/projected/9e4abac5-542b-4c97-9875-df2a128c7d61-kube-api-access-nfzb2\") pod \"ingress-canary-zrswq\" (UID: \"9e4abac5-542b-4c97-9875-df2a128c7d61\") " pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.440194 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.440113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrswq"
Sep 17 13:56:33.779946 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.779917    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrswq"]
Sep 17 13:56:33.783319 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:56:33.783295    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e4abac5_542b_4c97_9875_df2a128c7d61.slice/crio-aeee5e3409d990dfbb9cc11c825891579a7dbc42b86498e3b57db4cd7bcdea9a WatchSource:0}: Error finding container aeee5e3409d990dfbb9cc11c825891579a7dbc42b86498e3b57db4cd7bcdea9a: Status 404 returned error can't find the container with id aeee5e3409d990dfbb9cc11c825891579a7dbc42b86498e3b57db4cd7bcdea9a
Sep 17 13:56:33.900670 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:33.900622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:33.900854 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:33.900792    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:33.900900 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:33.900857    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:56:34.900842086 +0000 UTC m=+37.137719540 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:34.337589 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:34.337553    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrswq" event={"ID":"9e4abac5-542b-4c97-9875-df2a128c7d61","Type":"ContainerStarted","Data":"aeee5e3409d990dfbb9cc11c825891579a7dbc42b86498e3b57db4cd7bcdea9a"}
Sep 17 13:56:34.341392 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:34.341364    2570 generic.go:356] "Generic (PLEG): container finished" podID="eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69" containerID="d479a160c667c429a68f7b5221e3137caf5984417df8ac93abf618bc8da5071d" exitCode=0
Sep 17 13:56:34.341526 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:34.341404    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerDied","Data":"d479a160c667c429a68f7b5221e3137caf5984417df8ac93abf618bc8da5071d"}
Sep 17 13:56:34.906957 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:34.906913    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:34.907147 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:34.907084    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:34.907212 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:34.907160    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:56:36.907139832 +0000 UTC m=+39.144017290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:35.347961 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:35.347889    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" event={"ID":"eff4a2e2-f7d1-4fc8-a881-ebad6be2ae69","Type":"ContainerStarted","Data":"bfd1418d06cdde729511d1ab46d43ee640fc55a866fc4d81edb5a8f1e0a08f4a"}
Sep 17 13:56:35.377803 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:35.377758    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qtfw6" podStartSLOduration=8.25540157 podStartE2EDuration="37.377727505s" totalImagesPullingTime="29.122325935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:35.375285551 +0000 UTC m=+37.612163028" watchObservedRunningTime="2026-09-17 13:56:35.377727505 +0000 UTC m=+37.614604997"
Sep 17 13:56:36.923652 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:36.923464    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:36.924006 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:36.923606    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:36.924006 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:36.923784    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:56:40.923769197 +0000 UTC m=+43.160646651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:37.353818 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:37.353747    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrswq" event={"ID":"9e4abac5-542b-4c97-9875-df2a128c7d61","Type":"ContainerStarted","Data":"b5b3645b0b51bf43acce24e36edaee8f51420aa50f56631aa7f6af236a948a3d"}
Sep 17 13:56:37.373411 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:37.373355    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zrswq" podStartSLOduration=1.8816235510000001 podStartE2EDuration="4.373339209s" totalImagesPullingTime="2.491715658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:56:37.372649757 +0000 UTC m=+39.609527235" watchObservedRunningTime="2026-09-17 13:56:37.373339209 +0000 UTC m=+39.610216685"
Sep 17 13:56:40.950101 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:40.950070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:40.950465 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:40.950197    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:40.950465 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:40.950244    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:56:48.950231538 +0000 UTC m=+51.187108993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:49.005987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:49.005953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:56:49.006455 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:49.006082    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:56:49.006455 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:56:49.006134    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:57:05.006120465 +0000 UTC m=+67.242997919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:56:59.332566 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:56:59.332536    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lq2d"
Sep 17 13:57:04.005101 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.005062    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:57:04.005101 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.005107    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:57:04.008494 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.008473    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 13:57:04.008748 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.008714    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 13:57:04.016173 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:04.016155    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 13:57:04.016260 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:04.016226    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs podName:b93295b6-74b9-4536-a67f-532b4c8f49b8 nodeName:}" failed. No retries permitted until 2026-09-17 13:58:08.016207342 +0000 UTC m=+130.253084798 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs") pod "network-metrics-daemon-c4v66" (UID: "b93295b6-74b9-4536-a67f-532b4c8f49b8") : secret "metrics-daemon-secret" not found
Sep 17 13:57:04.019284 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.019264    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46b9c53f-aba0-48fc-b364-d9c3a167f498-original-pull-secret\") pod \"global-pull-secret-syncer-vpfvq\" (UID: \"46b9c53f-aba0-48fc-b364-d9c3a167f498\") " pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:57:04.089168 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.089131    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vpfvq"
Sep 17 13:57:04.106029 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.105998    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:57:04.109258 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.109238    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 13:57:04.120105 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.120084    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 13:57:04.130706 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.130682    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jf6f\" (UniqueName: \"kubernetes.io/projected/021db228-863a-4003-8cfc-218ec61e30c5-kube-api-access-4jf6f\") pod \"network-check-target-9lmwq\" (UID: \"021db228-863a-4003-8cfc-218ec61e30c5\") " pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:57:04.378969 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.378894    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vpfvq"]
Sep 17 13:57:04.381921 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:04.381893    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b9c53f_aba0_48fc_b364_d9c3a167f498.slice/crio-fb42f6936b8aa517396f2c01ac6d98fb398fc5fc560d4d21b33579ae71a2d1aa WatchSource:0}: Error finding container fb42f6936b8aa517396f2c01ac6d98fb398fc5fc560d4d21b33579ae71a2d1aa: Status 404 returned error can't find the container with id fb42f6936b8aa517396f2c01ac6d98fb398fc5fc560d4d21b33579ae71a2d1aa
Sep 17 13:57:04.396514 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.396495    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cpdmf\""
Sep 17 13:57:04.404103 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.404087    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:57:04.416000 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.415976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpfvq" event={"ID":"46b9c53f-aba0-48fc-b364-d9c3a167f498","Type":"ContainerStarted","Data":"fb42f6936b8aa517396f2c01ac6d98fb398fc5fc560d4d21b33579ae71a2d1aa"}
Sep 17 13:57:04.709603 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:04.709575    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9lmwq"]
Sep 17 13:57:04.712186 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:04.712162    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021db228_863a_4003_8cfc_218ec61e30c5.slice/crio-fdf466dd2ca0685da4df9b977da6b7bd1a879837ebe135f87f298dcf92a21cf7 WatchSource:0}: Error finding container fdf466dd2ca0685da4df9b977da6b7bd1a879837ebe135f87f298dcf92a21cf7: Status 404 returned error can't find the container with id fdf466dd2ca0685da4df9b977da6b7bd1a879837ebe135f87f298dcf92a21cf7
Sep 17 13:57:05.011268 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:05.011189    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:05.011601 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:05.011302    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 13:57:05.011601 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:05.011357    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls podName:9c05b88d-95b9-4735-bfd6-f896023da4be nodeName:}" failed. No retries permitted until 2026-09-17 13:57:37.011343157 +0000 UTC m=+99.248220615 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls") pod "dns-default-2j5sd" (UID: "9c05b88d-95b9-4735-bfd6-f896023da4be") : secret "dns-default-metrics-tls" not found
Sep 17 13:57:05.420191 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:05.420149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9lmwq" event={"ID":"021db228-863a-4003-8cfc-218ec61e30c5","Type":"ContainerStarted","Data":"fdf466dd2ca0685da4df9b977da6b7bd1a879837ebe135f87f298dcf92a21cf7"}
Sep 17 13:57:09.433136 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:09.433088    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vpfvq" event={"ID":"46b9c53f-aba0-48fc-b364-d9c3a167f498","Type":"ContainerStarted","Data":"68a5558c173462c695bea009dd3772116da0332a028020b7b5e3a9f1f2c68e8c"}
Sep 17 13:57:09.434566 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:09.434544    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9lmwq" event={"ID":"021db228-863a-4003-8cfc-218ec61e30c5","Type":"ContainerStarted","Data":"9a53da2b816a61ff2e0a360c108b785b0f6613205de1328f2c384b812ec8cd19"}
Sep 17 13:57:09.434679 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:09.434667    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:57:09.449991 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:09.449954    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vpfvq" podStartSLOduration=66.678141716 podStartE2EDuration="1m11.449940803s" totalImagesPullingTime="4.771799087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:09.449824912 +0000 UTC m=+71.686702390" watchObservedRunningTime="2026-09-17 13:57:09.449940803 +0000 UTC m=+71.686818281"
Sep 17 13:57:09.469485 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:09.469441    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9lmwq" podStartSLOduration=67.033745599 podStartE2EDuration="1m11.469430807s" totalImagesPullingTime="4.435685208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:09.468175258 +0000 UTC m=+71.705052735" watchObservedRunningTime="2026-09-17 13:57:09.469430807 +0000 UTC m=+71.706308283"
Sep 17 13:57:12.342868 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.342836    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-qssf9"]
Sep 17 13:57:12.344708 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.344693    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.352890 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.352872    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 13:57:12.354449 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.354430    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 17 13:57:12.354449 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.354444    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 13:57:12.354607 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.354443    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-wxk7n\""
Sep 17 13:57:12.354607 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.354431    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 17 13:57:12.359986 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.359969    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 17 13:57:12.374520 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.374500    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qssf9"]
Sep 17 13:57:12.468672 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.468845 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-snapshots\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.468845 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468727    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-service-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.468845 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468777    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqs22\" (UniqueName: \"kubernetes.io/projected/5f60e758-62a8-4f6b-bc63-7366f201fa4c-kube-api-access-vqs22\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.468972 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468858    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-tmp\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.468972 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.468891    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f60e758-62a8-4f6b-bc63-7366f201fa4c-serving-cert\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569243 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-tmp\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569354 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f60e758-62a8-4f6b-bc63-7366f201fa4c-serving-cert\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569354 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569275    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569354 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569304    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-snapshots\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569354 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569323    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-service-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569554 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569446    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqs22\" (UniqueName: \"kubernetes.io/projected/5f60e758-62a8-4f6b-bc63-7366f201fa4c-kube-api-access-vqs22\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.569669 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.569654    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-tmp\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.570046 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.570023    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5f60e758-62a8-4f6b-bc63-7366f201fa4c-snapshots\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.570143 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.570023    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-service-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.570518 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.570499    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f60e758-62a8-4f6b-bc63-7366f201fa4c-trusted-ca-bundle\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.571772 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.571754    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f60e758-62a8-4f6b-bc63-7366f201fa4c-serving-cert\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.586552 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.586527    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqs22\" (UniqueName: \"kubernetes.io/projected/5f60e758-62a8-4f6b-bc63-7366f201fa4c-kube-api-access-vqs22\") pod \"insights-operator-5844b856db-qssf9\" (UID: \"5f60e758-62a8-4f6b-bc63-7366f201fa4c\") " pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.652633 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.652615    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-qssf9"
Sep 17 13:57:12.965167 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:12.965096    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-qssf9"]
Sep 17 13:57:12.968200 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:12.968174    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f60e758_62a8_4f6b_bc63_7366f201fa4c.slice/crio-809f209a82b205b354cd43c752bc2d2f48c6a9bc7d8414074f0a4ac91ac3921d WatchSource:0}: Error finding container 809f209a82b205b354cd43c752bc2d2f48c6a9bc7d8414074f0a4ac91ac3921d: Status 404 returned error can't find the container with id 809f209a82b205b354cd43c752bc2d2f48c6a9bc7d8414074f0a4ac91ac3921d
Sep 17 13:57:13.445301 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:13.445278    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qssf9" event={"ID":"5f60e758-62a8-4f6b-bc63-7366f201fa4c","Type":"ContainerStarted","Data":"809f209a82b205b354cd43c752bc2d2f48c6a9bc7d8414074f0a4ac91ac3921d"}
Sep 17 13:57:15.452721 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.452682    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qssf9" event={"ID":"5f60e758-62a8-4f6b-bc63-7366f201fa4c","Type":"ContainerStarted","Data":"c771c3946eee6bc7988428ad5a5c4678ec600dcfebbedef2d2bd5bfcf70dc994"}
Sep 17 13:57:15.478267 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.478222    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-qssf9" podStartSLOduration=1.672003256 podStartE2EDuration="3.478207898s" totalImagesPullingTime="1.806204642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:15.47777134 +0000 UTC m=+77.714648817" watchObservedRunningTime="2026-09-17 13:57:15.478207898 +0000 UTC m=+77.715085375"
Sep 17 13:57:15.960434 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.960400    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-22gkb"]
Sep 17 13:57:15.965037 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.965009    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:15.968517 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.968493    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 13:57:15.968645 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.968503    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-twkdq\""
Sep 17 13:57:15.969870 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.969851    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 13:57:15.998990 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:15.998968    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-22gkb"]
Sep 17 13:57:16.092275 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.092248    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.092275 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.092277    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.092441 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.092295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec40f70f-5ed2-4631-b3dc-bf047ad23703-data-volume\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.092441 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.092322    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec40f70f-5ed2-4631-b3dc-bf047ad23703-crio-socket\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.092441 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.092339    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ksrv\" (UniqueName: \"kubernetes.io/projected/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-api-access-9ksrv\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192610 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192585    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192610 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192610    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192794 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192630    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec40f70f-5ed2-4631-b3dc-bf047ad23703-data-volume\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192794 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec40f70f-5ed2-4631-b3dc-bf047ad23703-crio-socket\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192794 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ksrv\" (UniqueName: \"kubernetes.io/projected/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-api-access-9ksrv\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192794 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.192725    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:16.192949 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.192804    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:16.692788698 +0000 UTC m=+78.929666169 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:16.192949 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192819    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ec40f70f-5ed2-4631-b3dc-bf047ad23703-crio-socket\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.192949 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.192862    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 13:57:16.192949 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.192926    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:16.692915286 +0000 UTC m=+78.929792741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : configmap "kube-rbac-proxy" not found
Sep 17 13:57:16.193077 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.192976    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec40f70f-5ed2-4631-b3dc-bf047ad23703-data-volume\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.202225 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.202206    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ksrv\" (UniqueName: \"kubernetes.io/projected/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-api-access-9ksrv\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.695623 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.695587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.695623 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:16.695622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:16.696045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.695693    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 13:57:16.696045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.695724    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:16.696045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.695762    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:17.695748995 +0000 UTC m=+79.932626449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : configmap "kube-rbac-proxy" not found
Sep 17 13:57:16.696045 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:16.695781    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:17.695768725 +0000 UTC m=+79.932646179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:17.218776 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.218746    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"]
Sep 17 13:57:17.220612 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.220598    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.227844 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.227824    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 17 13:57:17.227962 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.227858    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 17 13:57:17.228500 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.228485    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 17 13:57:17.228595 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.228583    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5qhrk\""
Sep 17 13:57:17.236231 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.236216    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"]
Sep 17 13:57:17.299906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.299880    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlqx\" (UniqueName: \"kubernetes.io/projected/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-kube-api-access-9tlqx\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.300020 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.299945    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.322099 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.322077    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"]
Sep 17 13:57:17.323938 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.323923    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"
Sep 17 13:57:17.327168 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.327144    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-z8klt\""
Sep 17 13:57:17.327264 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.327187    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 17 13:57:17.327808 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.327791    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 17 13:57:17.344197 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.344180    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"]
Sep 17 13:57:17.400364 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.400334    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.400487 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.400397    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5wl6\" (UniqueName: \"kubernetes.io/projected/fe2e7469-1275-48b1-9b3a-84e1455e35f9-kube-api-access-k5wl6\") pod \"volume-data-source-validator-86695b7cc7-m82gp\" (UID: \"fe2e7469-1275-48b1-9b3a-84e1455e35f9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"
Sep 17 13:57:17.400487 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.400431    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlqx\" (UniqueName: \"kubernetes.io/projected/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-kube-api-access-9tlqx\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.400596 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.400483    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:17.400596 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.400549    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:17.900530702 +0000 UTC m=+80.137408157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:17.410256 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.410234    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlqx\" (UniqueName: \"kubernetes.io/projected/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-kube-api-access-9tlqx\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.501518 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.501457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5wl6\" (UniqueName: \"kubernetes.io/projected/fe2e7469-1275-48b1-9b3a-84e1455e35f9-kube-api-access-k5wl6\") pod \"volume-data-source-validator-86695b7cc7-m82gp\" (UID: \"fe2e7469-1275-48b1-9b3a-84e1455e35f9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"
Sep 17 13:57:17.512764 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.512744    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5wl6\" (UniqueName: \"kubernetes.io/projected/fe2e7469-1275-48b1-9b3a-84e1455e35f9-kube-api-access-k5wl6\") pod \"volume-data-source-validator-86695b7cc7-m82gp\" (UID: \"fe2e7469-1275-48b1-9b3a-84e1455e35f9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"
Sep 17 13:57:17.632818 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.632793    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"
Sep 17 13:57:17.703197 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.703171    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:17.703537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.703204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:17.703537 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.703282    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 13:57:17.703537 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.703335    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:19.703320885 +0000 UTC m=+81.940198340 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : configmap "kube-rbac-proxy" not found
Sep 17 13:57:17.703537 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.703340    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:17.703537 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.703410    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:19.703391275 +0000 UTC m=+81.940268751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:17.905497 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.905471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:17.905619 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.905573    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:17.905658 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:17.905621    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:18.905608249 +0000 UTC m=+81.142485704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:17.917625 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:17.917602    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp"]
Sep 17 13:57:17.920521 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:17.920496    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2e7469_1275_48b1_9b3a_84e1455e35f9.slice/crio-aeb077af51f6e7c816a0fca56bb26fe378c2c3f37fa66a2a669c37d914c4ca5e WatchSource:0}: Error finding container aeb077af51f6e7c816a0fca56bb26fe378c2c3f37fa66a2a669c37d914c4ca5e: Status 404 returned error can't find the container with id aeb077af51f6e7c816a0fca56bb26fe378c2c3f37fa66a2a669c37d914c4ca5e
Sep 17 13:57:18.073919 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:18.073897    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sw8gq_10ac6fa6-352e-4052-8d50-dfc868b889f0/dns-node-resolver/0.log"
Sep 17 13:57:18.460745 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:18.460687    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp" event={"ID":"fe2e7469-1275-48b1-9b3a-84e1455e35f9","Type":"ContainerStarted","Data":"aeb077af51f6e7c816a0fca56bb26fe378c2c3f37fa66a2a669c37d914c4ca5e"}
Sep 17 13:57:18.666342 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:18.666318    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dbsps_ddc3a6a1-6e86-47ca-811a-4334fbcffbbb/node-ca/0.log"
Sep 17 13:57:18.911107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:18.911072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:18.911507 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:18.911226    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:18.911507 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:18.911336    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:20.911320324 +0000 UTC m=+83.148197779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:19.464903 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:19.464872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp" event={"ID":"fe2e7469-1275-48b1-9b3a-84e1455e35f9","Type":"ContainerStarted","Data":"9fdf2deaca773fc8c4020af4d53ee447a309ae37c9f5ab7f665d8faca2dc7ebd"}
Sep 17 13:57:19.482871 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:19.482821    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-m82gp" podStartSLOduration=1.10453146 podStartE2EDuration="2.482804686s" totalImagesPullingTime="1.378273226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:19.482316252 +0000 UTC m=+81.719193729" watchObservedRunningTime="2026-09-17 13:57:19.482804686 +0000 UTC m=+81.719682164"
Sep 17 13:57:19.673169 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:19.673146    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrswq_9e4abac5-542b-4c97-9875-df2a128c7d61/serve-healthcheck-canary/0.log"
Sep 17 13:57:19.718032 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:19.718003    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:19.718159 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:19.718040    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:19.718159 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:19.718146    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:19.718244 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:19.718162    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 13:57:19.718244 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:19.718201    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:23.718188608 +0000 UTC m=+85.955066062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:19.718244 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:19.718214    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:23.718208281 +0000 UTC m=+85.955085736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : configmap "kube-rbac-proxy" not found
Sep 17 13:57:20.927168 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:20.927129    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:20.927551 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:20.927286    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:20.927551 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:20.927360    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:24.927345798 +0000 UTC m=+87.164223253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:23.745651 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:23.745621    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:23.745651 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:23.745651    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:23.746116 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:23.745759    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 13:57:23.746116 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:23.745809    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:31.745796425 +0000 UTC m=+93.982673879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : configmap "kube-rbac-proxy" not found
Sep 17 13:57:23.746116 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:23.745809    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:23.746116 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:23.745871    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:31.745853927 +0000 UTC m=+93.982731399 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:24.953397 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:24.953360    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:24.953755 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:24.953474    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:24.953755 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:24.953527    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:32.953513859 +0000 UTC m=+95.190391314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:27.465966 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.465937    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"]
Sep 17 13:57:27.470188 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.470173    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"
Sep 17 13:57:27.473031 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.473010    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 17 13:57:27.473152 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.473039    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 17 13:57:27.473152 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.473108    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-sq57j\""
Sep 17 13:57:27.478052 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.478035    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"]
Sep 17 13:57:27.571113 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.571085    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f8g2\" (UniqueName: \"kubernetes.io/projected/973b5a26-c959-4633-9fdc-ce81acee35db-kube-api-access-6f8g2\") pod \"migrator-8b4d697c5-kqb9s\" (UID: \"973b5a26-c959-4633-9fdc-ce81acee35db\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"
Sep 17 13:57:27.672361 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.672326    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f8g2\" (UniqueName: \"kubernetes.io/projected/973b5a26-c959-4633-9fdc-ce81acee35db-kube-api-access-6f8g2\") pod \"migrator-8b4d697c5-kqb9s\" (UID: \"973b5a26-c959-4633-9fdc-ce81acee35db\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"
Sep 17 13:57:27.681804 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.681786    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f8g2\" (UniqueName: \"kubernetes.io/projected/973b5a26-c959-4633-9fdc-ce81acee35db-kube-api-access-6f8g2\") pod \"migrator-8b4d697c5-kqb9s\" (UID: \"973b5a26-c959-4633-9fdc-ce81acee35db\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"
Sep 17 13:57:27.778483 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:27.778434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"
Sep 17 13:57:28.052357 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:28.052286    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s"]
Sep 17 13:57:28.055127 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:28.055098    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973b5a26_c959_4633_9fdc_ce81acee35db.slice/crio-53f8eb49f95b85cd4b09717eaf84addf52eea1c0f0e09a69623f17aa0bc43680 WatchSource:0}: Error finding container 53f8eb49f95b85cd4b09717eaf84addf52eea1c0f0e09a69623f17aa0bc43680: Status 404 returned error can't find the container with id 53f8eb49f95b85cd4b09717eaf84addf52eea1c0f0e09a69623f17aa0bc43680
Sep 17 13:57:28.488653 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:28.488615    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s" event={"ID":"973b5a26-c959-4633-9fdc-ce81acee35db","Type":"ContainerStarted","Data":"53f8eb49f95b85cd4b09717eaf84addf52eea1c0f0e09a69623f17aa0bc43680"}
Sep 17 13:57:30.494948 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:30.494910    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s" event={"ID":"973b5a26-c959-4633-9fdc-ce81acee35db","Type":"ContainerStarted","Data":"0ddac8440da9cbf6c7cb7c5a4e0384f4cbc40645a29e5aa49784e36e7779d1b7"}
Sep 17 13:57:30.495334 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:30.494972    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s" event={"ID":"973b5a26-c959-4633-9fdc-ce81acee35db","Type":"ContainerStarted","Data":"74ff5acd64082b7f4ab387e7bf6e355ff1301d82e5c05caa0c1d4898cab6fb3a"}
Sep 17 13:57:30.515794 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:30.515753    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-kqb9s" podStartSLOduration=2.090587978 podStartE2EDuration="3.515717051s" totalImagesPullingTime="1.425129073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:30.515251133 +0000 UTC m=+92.752128610" watchObservedRunningTime="2026-09-17 13:57:30.515717051 +0000 UTC m=+92.752594527"
Sep 17 13:57:31.804457 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:31.804427    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:31.804457 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:31.804459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:31.804918 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:31.804575    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:31.804918 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:31.804640    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls podName:ec40f70f-5ed2-4631-b3dc-bf047ad23703 nodeName:}" failed. No retries permitted until 2026-09-17 13:57:47.804624421 +0000 UTC m=+110.041501879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls") pod "insights-runtime-extractor-22gkb" (UID: "ec40f70f-5ed2-4631-b3dc-bf047ad23703") : secret "insights-runtime-extractor-tls" not found
Sep 17 13:57:31.804997 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:31.804937    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ec40f70f-5ed2-4631-b3dc-bf047ad23703-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:33.011302 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:33.011265    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:33.011707 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:33.011417    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 17 13:57:33.011707 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:57:33.011476    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls podName:0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e nodeName:}" failed. No retries permitted until 2026-09-17 13:57:49.011462766 +0000 UTC m=+111.248340221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-55gzf" (UID: "0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e") : secret "samples-operator-tls" not found
Sep 17 13:57:37.041520 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.041492    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:37.043937 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.043916    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c05b88d-95b9-4735-bfd6-f896023da4be-metrics-tls\") pod \"dns-default-2j5sd\" (UID: \"9c05b88d-95b9-4735-bfd6-f896023da4be\") " pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:37.055978 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.055962    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f8wrh\""
Sep 17 13:57:37.063701 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.063687    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:37.338314 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.338194    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2j5sd"]
Sep 17 13:57:37.340692 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:37.340671    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c05b88d_95b9_4735_bfd6_f896023da4be.slice/crio-5e8ceae1530235d45292f26006d164ff820cb1bb21fdaad10cbb21baf334ef03 WatchSource:0}: Error finding container 5e8ceae1530235d45292f26006d164ff820cb1bb21fdaad10cbb21baf334ef03: Status 404 returned error can't find the container with id 5e8ceae1530235d45292f26006d164ff820cb1bb21fdaad10cbb21baf334ef03
Sep 17 13:57:37.516598 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:37.516570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2j5sd" event={"ID":"9c05b88d-95b9-4735-bfd6-f896023da4be","Type":"ContainerStarted","Data":"5e8ceae1530235d45292f26006d164ff820cb1bb21fdaad10cbb21baf334ef03"}
Sep 17 13:57:39.524537 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:39.524470    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2j5sd" event={"ID":"9c05b88d-95b9-4735-bfd6-f896023da4be","Type":"ContainerStarted","Data":"4d17fccf85a74b84eb1281cf5dc194d7e15cde833c99b9bbd9abb155204f807c"}
Sep 17 13:57:39.524906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:39.524547    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2j5sd" event={"ID":"9c05b88d-95b9-4735-bfd6-f896023da4be","Type":"ContainerStarted","Data":"1627990814c108b825d658635a35fc9727b2d91eaf089c1a808b1724051bab4a"}
Sep 17 13:57:39.524906 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:39.524624    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:39.550267 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:39.550223    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2j5sd" podStartSLOduration=64.631743386 podStartE2EDuration="1m6.550210563s" totalImagesPullingTime="1.918467177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:39.548961897 +0000 UTC m=+101.785839376" watchObservedRunningTime="2026-09-17 13:57:39.550210563 +0000 UTC m=+101.787088040"
Sep 17 13:57:40.439662 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:40.439626    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9lmwq"
Sep 17 13:57:47.813634 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:47.813588    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:47.816087 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:47.816063    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ec40f70f-5ed2-4631-b3dc-bf047ad23703-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22gkb\" (UID: \"ec40f70f-5ed2-4631-b3dc-bf047ad23703\") " pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:48.074484 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:48.074392    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-22gkb"
Sep 17 13:57:48.365029 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:48.365003    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-22gkb"]
Sep 17 13:57:48.365489 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:48.365463    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec40f70f_5ed2_4631_b3dc_bf047ad23703.slice/crio-3134675e58b18cf255bbfe737809e58816717449ab96a7eff6648ab81fe97508 WatchSource:0}: Error finding container 3134675e58b18cf255bbfe737809e58816717449ab96a7eff6648ab81fe97508: Status 404 returned error can't find the container with id 3134675e58b18cf255bbfe737809e58816717449ab96a7eff6648ab81fe97508
Sep 17 13:57:48.552695 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:48.552668    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22gkb" event={"ID":"ec40f70f-5ed2-4631-b3dc-bf047ad23703","Type":"ContainerStarted","Data":"121f14957c7502e238f75ca6695ad1555a8e7473b54cb06d44a44cd9c83db9b8"}
Sep 17 13:57:48.552852 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:48.552716    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22gkb" event={"ID":"ec40f70f-5ed2-4631-b3dc-bf047ad23703","Type":"ContainerStarted","Data":"3134675e58b18cf255bbfe737809e58816717449ab96a7eff6648ab81fe97508"}
Sep 17 13:57:49.022119 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.022088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:49.024494 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.024473    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-55gzf\" (UID: \"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:49.029346 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.029329    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"
Sep 17 13:57:49.298179 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.298110    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf"]
Sep 17 13:57:49.530997 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.530973    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2j5sd"
Sep 17 13:57:49.563015 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:49.562915    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf" event={"ID":"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e","Type":"ContainerStarted","Data":"acd38095cabbc95d9de04929618d0a460bf3ca5d1c6bc4bd01aacdf079d1cb31"}
Sep 17 13:57:50.088872 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.088829    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"]
Sep 17 13:57:50.091055 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.091023    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.093518 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.093490    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"]
Sep 17 13:57:50.094408 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.094175    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mpsxs\""
Sep 17 13:57:50.095713 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.095689    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 17 13:57:50.095881 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.095715    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 17 13:57:50.095881 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.095701    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.113796 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.113770    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"]
Sep 17 13:57:50.113948 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.113932    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 13:57:50.114935 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.114912    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 13:57:50.115072 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.114987    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 13:57:50.115261 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.115238    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 13:57:50.116533 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.116513    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 13:57:50.116803 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.116787    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 13:57:50.118931 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.118912    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 13:57:50.125466 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.125446    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"]
Sep 17 13:57:50.228473 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.228445    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-844f746c6f-br9bn"]
Sep 17 13:57:50.230645 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.230624    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.232107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.231975    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c31924-7197-4f60-bd48-191ef604fec6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232026    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhfbd\" (UniqueName: \"kubernetes.io/projected/73c31924-7197-4f60-bd48-191ef604fec6-kube-api-access-hhfbd\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232107 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232352 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232124    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.232352 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232150    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232352 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232184    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.232352 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.232234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.234565 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.234545    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 13:57:50.234854 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.234834    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 13:57:50.234936 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.234911    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cjxbg\""
Sep 17 13:57:50.240677 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.240480    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 13:57:50.245890 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.245871    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 13:57:50.248762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.248717    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-844f746c6f-br9bn"]
Sep 17 13:57:50.333573 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333536    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhfbd\" (UniqueName: \"kubernetes.io/projected/73c31924-7197-4f60-bd48-191ef604fec6-kube-api-access-hhfbd\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333582    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-bound-sa-token\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333619    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333667    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-tls\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333692    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.333762 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-certificates\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-installation-pull-secrets\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333808    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrgtt\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-kube-api-access-qrgtt\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333902    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a31d439-e8b3-44e8-85e3-de77cf76ec55-ca-trust-extracted\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-trusted-ca\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.333975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c31924-7197-4f60-bd48-191ef604fec6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.334116 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.334000    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-image-registry-private-configuration\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.335989 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.335961    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.336101 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.336047    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c31924-7197-4f60-bd48-191ef604fec6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.337405 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.337365    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-ca\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.338577 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.338536    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.339482 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.339412    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.339573 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.339546    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-lwfc4\" (UID: \"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.340629 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.340608    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73c31924-7197-4f60-bd48-191ef604fec6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.346373 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.346352    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhfbd\" (UniqueName: \"kubernetes.io/projected/73c31924-7197-4f60-bd48-191ef604fec6-kube-api-access-hhfbd\") pod \"cluster-proxy-proxy-agent-844bc7fb65-dmdmx\" (UID: \"73c31924-7197-4f60-bd48-191ef604fec6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.405303 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.405176    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"
Sep 17 13:57:50.419534 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.419504    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435309    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrgtt\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-kube-api-access-qrgtt\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435384    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a31d439-e8b3-44e8-85e3-de77cf76ec55-ca-trust-extracted\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-trusted-ca\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-image-registry-private-configuration\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435498    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-bound-sa-token\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-tls\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-certificates\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.435998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.435585    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-installation-pull-secrets\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.436438 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.436409    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8a31d439-e8b3-44e8-85e3-de77cf76ec55-ca-trust-extracted\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.437779 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.437212    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-trusted-ca\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.437779 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.437712    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-certificates\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.439307 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.439289    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-registry-tls\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.467458 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.467383    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-installation-pull-secrets\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.474713 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.474677    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrgtt\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-kube-api-access-qrgtt\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.475474 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.475455    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a31d439-e8b3-44e8-85e3-de77cf76ec55-bound-sa-token\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.475853 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.475832    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8a31d439-e8b3-44e8-85e3-de77cf76ec55-image-registry-private-configuration\") pod \"image-registry-844f746c6f-br9bn\" (UID: \"8a31d439-e8b3-44e8-85e3-de77cf76ec55\") " pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.541482 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.541435    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:50.569279 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.569228    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22gkb" event={"ID":"ec40f70f-5ed2-4631-b3dc-bf047ad23703","Type":"ContainerStarted","Data":"1a813370bb29b8df7dbd4ed7181a494f813b48f5212a3fea789b762dc2f567b6"}
Sep 17 13:57:50.714673 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.714604    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4"]
Sep 17 13:57:50.739799 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.739777    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx"]
Sep 17 13:57:50.835615 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:50.835512    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-844f746c6f-br9bn"]
Sep 17 13:57:51.013053 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:51.013021    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf448c1_5c33_4e1b_9f66_45ce62ac1e1b.slice/crio-7cd6a36b91527d27b42a395e236572d5b362c9122cb38eb0b7ace8b9b04b4473 WatchSource:0}: Error finding container 7cd6a36b91527d27b42a395e236572d5b362c9122cb38eb0b7ace8b9b04b4473: Status 404 returned error can't find the container with id 7cd6a36b91527d27b42a395e236572d5b362c9122cb38eb0b7ace8b9b04b4473
Sep 17 13:57:51.013775 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:51.013722    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a31d439_e8b3_44e8_85e3_de77cf76ec55.slice/crio-f871a768208b1647d2d3746b33cbd66566d6a71b3ab4bc4fa086c93bcdbda36b WatchSource:0}: Error finding container f871a768208b1647d2d3746b33cbd66566d6a71b3ab4bc4fa086c93bcdbda36b: Status 404 returned error can't find the container with id f871a768208b1647d2d3746b33cbd66566d6a71b3ab4bc4fa086c93bcdbda36b
Sep 17 13:57:51.014484 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:51.014446    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73c31924_7197_4f60_bd48_191ef604fec6.slice/crio-83e64e2d77e9d71ff4a85b8fda6d7c1aceeb63ad8d97e66ba94320715ac01b19 WatchSource:0}: Error finding container 83e64e2d77e9d71ff4a85b8fda6d7c1aceeb63ad8d97e66ba94320715ac01b19: Status 404 returned error can't find the container with id 83e64e2d77e9d71ff4a85b8fda6d7c1aceeb63ad8d97e66ba94320715ac01b19
Sep 17 13:57:51.579947 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.579330    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-844f746c6f-br9bn" event={"ID":"8a31d439-e8b3-44e8-85e3-de77cf76ec55","Type":"ContainerStarted","Data":"f191e38992984d7950df6580b3570e6c064cb6d027c765978e7117765d7ca64f"}
Sep 17 13:57:51.579947 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.579390    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-844f746c6f-br9bn" event={"ID":"8a31d439-e8b3-44e8-85e3-de77cf76ec55","Type":"ContainerStarted","Data":"f871a768208b1647d2d3746b33cbd66566d6a71b3ab4bc4fa086c93bcdbda36b"}
Sep 17 13:57:51.579947 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.579807    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:57:51.590223 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.588377    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22gkb" event={"ID":"ec40f70f-5ed2-4631-b3dc-bf047ad23703","Type":"ContainerStarted","Data":"bb2f487a36f0ec45870df243ecc7ac9737298d4d48e34318675cff2b5aeffd57"}
Sep 17 13:57:51.595270 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.595236    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf" event={"ID":"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e","Type":"ContainerStarted","Data":"d8ea5a65ece788d5325f5fc2255f721ad27a084dc87d958c675a660d8d72b1cd"}
Sep 17 13:57:51.595386 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.595325    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf" event={"ID":"0c11fcfa-48fb-4a1f-8a3a-6b0ecf63848e","Type":"ContainerStarted","Data":"d77c9d01d1d32ffa8153ca151a7939a1b21caf4491d061efddea088f8c7e794e"}
Sep 17 13:57:51.597907 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.597861    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx" event={"ID":"73c31924-7197-4f60-bd48-191ef604fec6","Type":"ContainerStarted","Data":"83e64e2d77e9d71ff4a85b8fda6d7c1aceeb63ad8d97e66ba94320715ac01b19"}
Sep 17 13:57:51.600525 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.599512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4" event={"ID":"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b","Type":"ContainerStarted","Data":"7cd6a36b91527d27b42a395e236572d5b362c9122cb38eb0b7ace8b9b04b4473"}
Sep 17 13:57:51.616375 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.616047    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-844f746c6f-br9bn" podStartSLOduration=1.6160302340000001 podStartE2EDuration="1.616030234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:51.613777559 +0000 UTC m=+113.850655029" watchObservedRunningTime="2026-09-17 13:57:51.616030234 +0000 UTC m=+113.852907710"
Sep 17 13:57:51.642963 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.642875    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-22gkb" podStartSLOduration=33.695718844 podStartE2EDuration="36.642856057s" totalImagesPullingTime="2.947137213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 13:57:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:51.642353166 +0000 UTC m=+113.879230645" watchObservedRunningTime="2026-09-17 13:57:51.642856057 +0000 UTC m=+113.879733536"
Sep 17 13:57:51.673663 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:51.672533    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-55gzf" podStartSLOduration=32.597858561 podStartE2EDuration="34.67251552s" totalImagesPullingTime="2.074656959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:51.6706917 +0000 UTC m=+113.907569178" watchObservedRunningTime="2026-09-17 13:57:51.67251552 +0000 UTC m=+113.909392998"
Sep 17 13:57:55.619203 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.619171    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx" event={"ID":"73c31924-7197-4f60-bd48-191ef604fec6","Type":"ContainerStarted","Data":"ad028b5e1620c488cd16bacdb73a36ee67e1228e39ff848a6a4768da7db20ce5"}
Sep 17 13:57:55.619203 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.619210    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx" event={"ID":"73c31924-7197-4f60-bd48-191ef604fec6","Type":"ContainerStarted","Data":"c332c53b3ab4ad2c26bb7e1de361ad2a35d1ba8441a26b9c45f4391c71803138"}
Sep 17 13:57:55.619700 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.619223    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx" event={"ID":"73c31924-7197-4f60-bd48-191ef604fec6","Type":"ContainerStarted","Data":"54fd52bba5e4d2b77add16309ae99cefd92ad53d33b82414c3ec7ead35c72617"}
Sep 17 13:57:55.620722 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.620701    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4" event={"ID":"bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b","Type":"ContainerStarted","Data":"0945d9d61a672b2882c4098f31d4a6fa261d26bb5a0a8dfafba53ae1cd48bf98"}
Sep 17 13:57:55.645522 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.645469    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-844bc7fb65-dmdmx" podStartSLOduration=1.9905244359999998 podStartE2EDuration="5.645454242s" totalImagesPullingTime="3.654929806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:55.645225261 +0000 UTC m=+117.882102738" watchObservedRunningTime="2026-09-17 13:57:55.645454242 +0000 UTC m=+117.882331720"
Sep 17 13:57:55.667650 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:55.667587    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-lwfc4" podStartSLOduration=2.022199205 podStartE2EDuration="5.667565835s" totalImagesPullingTime="3.64536663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:57:55.665565453 +0000 UTC m=+117.902442929" watchObservedRunningTime="2026-09-17 13:57:55.667565835 +0000 UTC m=+117.904443315"
Sep 17 13:57:59.342073 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.342043    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qnc84"]
Sep 17 13:57:59.345004 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.344986    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.347873 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.347852    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 13:57:59.348273 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.348256    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 13:57:59.348331 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.348292    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 13:57:59.348545 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.348527    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 13:57:59.349127 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.349113    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 13:57:59.349198 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.349128    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 13:57:59.349247 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.349226    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s544l\""
Sep 17 13:57:59.513083 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513047    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-root\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513083 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgfd8\" (UniqueName: \"kubernetes.io/projected/a13ff726-d8ff-495c-887f-8d1a90599cde-kube-api-access-xgfd8\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513110    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-sys\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513126    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-metrics-client-ca\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513183    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513216    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-wtmp\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513239    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-textfile\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513263    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-tls\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.513316 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.513307    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.613758 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613649    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-wtmp\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.613758 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613692    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-textfile\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.613758 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-tls\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613779    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613810    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-root\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613839    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgfd8\" (UniqueName: \"kubernetes.io/projected/a13ff726-d8ff-495c-887f-8d1a90599cde-kube-api-access-xgfd8\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613860    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-wtmp\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613867    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-sys\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613923    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-sys\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613927    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-metrics-client-ca\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613920    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a13ff726-d8ff-495c-887f-8d1a90599cde-root\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614026 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.613995    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614467 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.614177    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-textfile\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614467 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.614452    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-metrics-client-ca\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.614599 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.614580    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-accelerators-collector-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.616331 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.616308    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-tls\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.616420 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.616360    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a13ff726-d8ff-495c-887f-8d1a90599cde-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.633415 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.633391    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgfd8\" (UniqueName: \"kubernetes.io/projected/a13ff726-d8ff-495c-887f-8d1a90599cde-kube-api-access-xgfd8\") pod \"node-exporter-qnc84\" (UID: \"a13ff726-d8ff-495c-887f-8d1a90599cde\") " pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.654482 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:57:59.654456    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qnc84"
Sep 17 13:57:59.666138 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:57:59.666111    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda13ff726_d8ff_495c_887f_8d1a90599cde.slice/crio-2d28eb3e14d5a67c0b773c41fdd43abbcdbf27b867936833ebceaef8090c7bf9 WatchSource:0}: Error finding container 2d28eb3e14d5a67c0b773c41fdd43abbcdbf27b867936833ebceaef8090c7bf9: Status 404 returned error can't find the container with id 2d28eb3e14d5a67c0b773c41fdd43abbcdbf27b867936833ebceaef8090c7bf9
Sep 17 13:58:00.640289 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:00.640257    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnc84" event={"ID":"a13ff726-d8ff-495c-887f-8d1a90599cde","Type":"ContainerStarted","Data":"2d28eb3e14d5a67c0b773c41fdd43abbcdbf27b867936833ebceaef8090c7bf9"}
Sep 17 13:58:01.644612 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:01.644576    2570 generic.go:356] "Generic (PLEG): container finished" podID="a13ff726-d8ff-495c-887f-8d1a90599cde" containerID="c6a7d7e61346a52e4eeb2d9302852a41b46e10b26ebc964f5cd71d0e2d44cb9c" exitCode=0
Sep 17 13:58:01.644612 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:01.644616    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnc84" event={"ID":"a13ff726-d8ff-495c-887f-8d1a90599cde","Type":"ContainerDied","Data":"c6a7d7e61346a52e4eeb2d9302852a41b46e10b26ebc964f5cd71d0e2d44cb9c"}
Sep 17 13:58:02.649661 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.649629    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnc84" event={"ID":"a13ff726-d8ff-495c-887f-8d1a90599cde","Type":"ContainerStarted","Data":"966e4cf77ef96171c7a6cfe7d1d7ab7e674b5f0f45e7280fb207421850c96292"}
Sep 17 13:58:02.650073 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.649676    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qnc84" event={"ID":"a13ff726-d8ff-495c-887f-8d1a90599cde","Type":"ContainerStarted","Data":"1186dfc32a04f12588e76e8fa6ade360b81fea699eb472a634e9bdb57fafa396"}
Sep 17 13:58:02.696194 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.696139    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qnc84" podStartSLOduration=2.453513963 podStartE2EDuration="3.696123331s" totalImagesPullingTime="1.171602577s" totalInitContainerRuntime="71.006791ms" isStatefulPod=true podCreationTimestamp="2026-09-17 13:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:58:02.69402377 +0000 UTC m=+124.930901247" watchObservedRunningTime="2026-09-17 13:58:02.696123331 +0000 UTC m=+124.933000808"
Sep 17 13:58:02.908621 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.908588    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:02.910682 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.910668    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:02.917486 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.917469    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 17 13:58:02.918194 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.918177    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 17 13:58:02.918287 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.918227    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 17 13:58:02.918475 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.918459    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 17 13:58:02.918908 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.918891    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 17 13:58:02.925464 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.925450    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 17 13:58:02.925623 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.925611    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lnt98\""
Sep 17 13:58:02.925765 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.925750    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 17 13:58:02.947883 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.947856    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 17 13:58:02.967361 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:02.967340    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:03.041424 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksvv\" (UniqueName: \"kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041595 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041595 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041453    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041595 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041521    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041595 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041580    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041834 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041600    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.041834 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.041620    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.142863 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.142825    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksvv\" (UniqueName: \"kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.142890    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.142917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.142948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.142978    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.143003    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143033 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.143033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143721 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.143698    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.143827 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.143783    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.144007 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.143986    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.144189 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.144169    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.145620 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.145601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.145696 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.145677    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.159564 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.159514    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksvv\" (UniqueName: \"kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv\") pod \"console-7b9fd97b58-s6ljb\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") " pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.219659 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.219623    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:03.503778 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.503690    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:03.506276 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:58:03.506243    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb90c75_3de5_4425_941e_921cb360b76e.slice/crio-f34523ababa46f8b8be81b68ce27381e397b33e52ffff0cf543291e4c7420f38 WatchSource:0}: Error finding container f34523ababa46f8b8be81b68ce27381e397b33e52ffff0cf543291e4c7420f38: Status 404 returned error can't find the container with id f34523ababa46f8b8be81b68ce27381e397b33e52ffff0cf543291e4c7420f38
Sep 17 13:58:03.654113 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:03.654077    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd97b58-s6ljb" event={"ID":"2bb90c75-3de5-4425-941e-921cb360b76e","Type":"ContainerStarted","Data":"f34523ababa46f8b8be81b68ce27381e397b33e52ffff0cf543291e4c7420f38"}
Sep 17 13:58:06.666048 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:06.666015    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd97b58-s6ljb" event={"ID":"2bb90c75-3de5-4425-941e-921cb360b76e","Type":"ContainerStarted","Data":"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"}
Sep 17 13:58:06.688013 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:06.687973    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b9fd97b58-s6ljb" podStartSLOduration=1.8650526680000001 podStartE2EDuration="4.687959934s" totalImagesPullingTime="2.822907266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:58:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:58:06.687110637 +0000 UTC m=+128.923988114" watchObservedRunningTime="2026-09-17 13:58:06.687959934 +0000 UTC m=+128.924837411"
Sep 17 13:58:06.805610 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:06.805578    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:08.082883 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.082845    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:58:08.085215 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.085189    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b93295b6-74b9-4536-a67f-532b4c8f49b8-metrics-certs\") pod \"network-metrics-daemon-c4v66\" (UID: \"b93295b6-74b9-4536-a67f-532b4c8f49b8\") " pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:58:08.286757 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.286706    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gs47f\""
Sep 17 13:58:08.293975 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.293953    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c4v66"
Sep 17 13:58:08.578209 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.578178    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c4v66"]
Sep 17 13:58:08.582339 ip-10-0-130-163 kubenswrapper[2570]: W0917 13:58:08.582314    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93295b6_74b9_4536_a67f_532b4c8f49b8.slice/crio-4853c1b5bd6894bf77bf10d99af544818df5ab7edef19bbcb81781a76630eedc WatchSource:0}: Error finding container 4853c1b5bd6894bf77bf10d99af544818df5ab7edef19bbcb81781a76630eedc: Status 404 returned error can't find the container with id 4853c1b5bd6894bf77bf10d99af544818df5ab7edef19bbcb81781a76630eedc
Sep 17 13:58:08.673987 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:08.673959    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c4v66" event={"ID":"b93295b6-74b9-4536-a67f-532b4c8f49b8","Type":"ContainerStarted","Data":"4853c1b5bd6894bf77bf10d99af544818df5ab7edef19bbcb81781a76630eedc"}
Sep 17 13:58:09.683650 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:09.683618    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c4v66" event={"ID":"b93295b6-74b9-4536-a67f-532b4c8f49b8","Type":"ContainerStarted","Data":"1f641babdec8581ce7867a0567e8142e35034cdbc2ef6317cdbe4dba0efefc78"}
Sep 17 13:58:10.688167 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:10.688135    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c4v66" event={"ID":"b93295b6-74b9-4536-a67f-532b4c8f49b8","Type":"ContainerStarted","Data":"3be8048057e172df7643749b08936e8932bd2af3d9970e7a7a27233fb27b3d42"}
Sep 17 13:58:10.713314 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:10.713271    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c4v66" podStartSLOduration=131.773126298 podStartE2EDuration="2m12.713256217s" totalImagesPullingTime="940.129919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 13:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 13:58:10.712212019 +0000 UTC m=+132.949089496" watchObservedRunningTime="2026-09-17 13:58:10.713256217 +0000 UTC m=+132.950133694"
Sep 17 13:58:12.607693 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:12.607668    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-844f746c6f-br9bn"
Sep 17 13:58:13.219959 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:13.219929    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:30.762269 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:30.762234    2570 generic.go:356] "Generic (PLEG): container finished" podID="5f60e758-62a8-4f6b-bc63-7366f201fa4c" containerID="c771c3946eee6bc7988428ad5a5c4678ec600dcfebbedef2d2bd5bfcf70dc994" exitCode=0
Sep 17 13:58:30.762627 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:30.762313    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qssf9" event={"ID":"5f60e758-62a8-4f6b-bc63-7366f201fa4c","Type":"ContainerDied","Data":"c771c3946eee6bc7988428ad5a5c4678ec600dcfebbedef2d2bd5bfcf70dc994"}
Sep 17 13:58:30.762679 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:30.762663    2570 scope.go:117] "RemoveContainer" containerID="c771c3946eee6bc7988428ad5a5c4678ec600dcfebbedef2d2bd5bfcf70dc994"
Sep 17 13:58:31.423590 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:31.423560    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2j5sd_9c05b88d-95b9-4735-bfd6-f896023da4be/dns/0.log"
Sep 17 13:58:31.621711 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:31.621681    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2j5sd_9c05b88d-95b9-4735-bfd6-f896023da4be/kube-rbac-proxy/0.log"
Sep 17 13:58:31.767547 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:31.767469    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-qssf9" event={"ID":"5f60e758-62a8-4f6b-bc63-7366f201fa4c","Type":"ContainerStarted","Data":"ad64b8df830ff0c5b4db9694870a342976119b784cb3d2f9232db7fbec6317e5"}
Sep 17 13:58:33.021307 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.021280    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sw8gq_10ac6fa6-352e-4052-8d50-dfc868b889f0/dns-node-resolver/0.log"
Sep 17 13:58:33.692651 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.692611    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b9fd97b58-s6ljb" podUID="2bb90c75-3de5-4425-941e-921cb360b76e" containerName="console" containerID="cri-o://6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3" gracePeriod=15
Sep 17 13:58:33.821615 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.821587    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrswq_9e4abac5-542b-4c97-9875-df2a128c7d61/serve-healthcheck-canary/0.log"
Sep 17 13:58:33.924777 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.924719    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:33.982790 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982701    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982790 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982771    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982809    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hksvv\" (UniqueName: \"kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982852    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982878    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982906    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.982998 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.982937    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca\") pod \"2bb90c75-3de5-4425-941e-921cb360b76e\" (UID: \"2bb90c75-3de5-4425-941e-921cb360b76e\") "
Sep 17 13:58:33.983317 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.983293    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config" (OuterVolumeSpecName: "console-config") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 13:58:33.983388 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.983308    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 13:58:33.983444 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.983411    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 13:58:33.983490 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.983443    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca" (OuterVolumeSpecName: "service-ca") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 13:58:33.985288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.985267    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv" (OuterVolumeSpecName: "kube-api-access-hksvv") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "kube-api-access-hksvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 13:58:33.985599 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.985581    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 13:58:33.985656 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:33.985600    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2bb90c75-3de5-4425-941e-921cb360b76e" (UID: "2bb90c75-3de5-4425-941e-921cb360b76e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 13:58:34.084288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084236    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-serving-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084282    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-console-config\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084288 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084295    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hksvv\" (UniqueName: \"kubernetes.io/projected/2bb90c75-3de5-4425-941e-921cb360b76e-kube-api-access-hksvv\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084781 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084307    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bb90c75-3de5-4425-941e-921cb360b76e-console-oauth-config\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084781 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084320    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-oauth-serving-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084781 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084330    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-trusted-ca-bundle\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.084781 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.084343    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bb90c75-3de5-4425-941e-921cb360b76e-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 13:58:34.779917 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.779883    2570 generic.go:356] "Generic (PLEG): container finished" podID="2bb90c75-3de5-4425-941e-921cb360b76e" containerID="6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3" exitCode=0
Sep 17 13:58:34.780109 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.779930    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd97b58-s6ljb"
Sep 17 13:58:34.780109 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.779966    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd97b58-s6ljb" event={"ID":"2bb90c75-3de5-4425-941e-921cb360b76e","Type":"ContainerDied","Data":"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"}
Sep 17 13:58:34.780109 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.780017    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd97b58-s6ljb" event={"ID":"2bb90c75-3de5-4425-941e-921cb360b76e","Type":"ContainerDied","Data":"f34523ababa46f8b8be81b68ce27381e397b33e52ffff0cf543291e4c7420f38"}
Sep 17 13:58:34.780109 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.780038    2570 scope.go:117] "RemoveContainer" containerID="6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"
Sep 17 13:58:34.788749 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.788713    2570 scope.go:117] "RemoveContainer" containerID="6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"
Sep 17 13:58:34.789105 ip-10-0-130-163 kubenswrapper[2570]: E0917 13:58:34.789079    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3\": container with ID starting with 6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3 not found: ID does not exist" containerID="6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"
Sep 17 13:58:34.789163 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.789117    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3"} err="failed to get container status \"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3\": rpc error: code = NotFound desc = could not find container \"6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3\": container with ID starting with 6752cc718a83305f57b75df5e786dfff7b19a51dab97868df0b7e08606bbb1e3 not found: ID does not exist"
Sep 17 13:58:34.800016 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.799995    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:34.804392 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:34.804370    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b9fd97b58-s6ljb"]
Sep 17 13:58:36.180280 ip-10-0-130-163 kubenswrapper[2570]: I0917 13:58:36.180248    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bb90c75-3de5-4425-941e-921cb360b76e" path="/var/lib/kubelet/pods/2bb90c75-3de5-4425-941e-921cb360b76e/volumes"
Sep 17 14:00:58.138868 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:00:58.138838    2570 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 14:04:32.055871 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.055837    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"]
Sep 17 14:04:32.056265 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.056124    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bb90c75-3de5-4425-941e-921cb360b76e" containerName="console"
Sep 17 14:04:32.056265 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.056140    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bb90c75-3de5-4425-941e-921cb360b76e" containerName="console"
Sep 17 14:04:32.056265 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.056194    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bb90c75-3de5-4425-941e-921cb360b76e" containerName="console"
Sep 17 14:04:32.059096 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.059082    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.067321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.067303    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gp7tv\""
Sep 17 14:04:32.067618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.067602    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 17 14:04:32.067618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.067608    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 17 14:04:32.067712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.067611    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 17 14:04:32.070910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.070889    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 17 14:04:32.078920 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.078900    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"]
Sep 17 14:04:32.234813 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.234778    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-manifests\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.234980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.234828    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-tmp\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.234980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.234845    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vchv4\" (UniqueName: \"kubernetes.io/projected/8e45c1c6-390e-4a0a-9132-a9c0162b868a-kube-api-access-vchv4\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.234980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.234888    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.234980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.234933    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.336633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.336071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-tmp\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.336633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.336121    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vchv4\" (UniqueName: \"kubernetes.io/projected/8e45c1c6-390e-4a0a-9132-a9c0162b868a-kube-api-access-vchv4\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.336633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.336161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.336633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.336218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.336633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.336276    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-manifests\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.337570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.337202    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-manifests\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.340569 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.340141    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e45c1c6-390e-4a0a-9132-a9c0162b868a-tmp\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.342870 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.342845    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.342948 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.342935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e45c1c6-390e-4a0a-9132-a9c0162b868a-webhook-cert\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.349049 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.349026    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vchv4\" (UniqueName: \"kubernetes.io/projected/8e45c1c6-390e-4a0a-9132-a9c0162b868a-kube-api-access-vchv4\") pod \"opendatahub-operator-controller-manager-fcc687555-fwkzn\" (UID: \"8e45c1c6-390e-4a0a-9132-a9c0162b868a\") " pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.368866 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.368846    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:32.650487 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.650459    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"]
Sep 17 14:04:32.651937 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:04:32.651909    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e45c1c6_390e_4a0a_9132_a9c0162b868a.slice/crio-0f8de59ff185a68c2f33292a36a116115be2dcd3e9a7793abac411041bf0da0d WatchSource:0}: Error finding container 0f8de59ff185a68c2f33292a36a116115be2dcd3e9a7793abac411041bf0da0d: Status 404 returned error can't find the container with id 0f8de59ff185a68c2f33292a36a116115be2dcd3e9a7793abac411041bf0da0d
Sep 17 14:04:32.653892 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:32.653873    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:04:33.001191 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:33.001151    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn" event={"ID":"8e45c1c6-390e-4a0a-9132-a9c0162b868a","Type":"ContainerStarted","Data":"0f8de59ff185a68c2f33292a36a116115be2dcd3e9a7793abac411041bf0da0d"}
Sep 17 14:04:36.014549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:36.014512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn" event={"ID":"8e45c1c6-390e-4a0a-9132-a9c0162b868a","Type":"ContainerStarted","Data":"f5411215821e45ff93ceeaa8e4f45555da5e58d8ff360071d65a71fe51a06baf"}
Sep 17 14:04:37.020216 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:37.020179    2570 generic.go:356] "Generic (PLEG): container finished" podID="8e45c1c6-390e-4a0a-9132-a9c0162b868a" containerID="f5411215821e45ff93ceeaa8e4f45555da5e58d8ff360071d65a71fe51a06baf" exitCode=0
Sep 17 14:04:37.020592 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:37.020267    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn" event={"ID":"8e45c1c6-390e-4a0a-9132-a9c0162b868a","Type":"ContainerDied","Data":"f5411215821e45ff93ceeaa8e4f45555da5e58d8ff360071d65a71fe51a06baf"}
Sep 17 14:04:38.026147 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:38.026116    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn" event={"ID":"8e45c1c6-390e-4a0a-9132-a9c0162b868a","Type":"ContainerStarted","Data":"afe712259c21c2566fe865c6ebf974526ce7c7d1040151e02e9ccccf304ee7d7"}
Sep 17 14:04:38.026577 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:38.026260    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:38.052636 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:38.052263    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn" podStartSLOduration=0.972295585 podStartE2EDuration="6.052242075s" totalImagesPullingTime="4.040189078s" totalInitContainerRuntime="1.039757412s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:04:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:04:38.049995456 +0000 UTC m=+520.286872934" watchObservedRunningTime="2026-09-17 14:04:38.052242075 +0000 UTC m=+520.289119555"
Sep 17 14:04:49.032590 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:49.032560    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fcc687555-fwkzn"
Sep 17 14:04:51.810773 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.810669    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"]
Sep 17 14:04:51.814008 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.813988    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.824891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.824873    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 17 14:04:51.826190 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.826170    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 14:04:51.832714 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.832698    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 17 14:04:51.832993 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.832974    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 14:04:51.851817 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.851791    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"]
Sep 17 14:04:51.879996 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.879975    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ad5755-2296-4448-a37c-821c89dc7adc-tls-certs\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.880118 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.880016    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjkl\" (UniqueName: \"kubernetes.io/projected/c0ad5755-2296-4448-a37c-821c89dc7adc-kube-api-access-bkjkl\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.880118 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.880039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ad5755-2296-4448-a37c-821c89dc7adc-tmp\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.981488 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.981458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ad5755-2296-4448-a37c-821c89dc7adc-tls-certs\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.981671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.981520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjkl\" (UniqueName: \"kubernetes.io/projected/c0ad5755-2296-4448-a37c-821c89dc7adc-kube-api-access-bkjkl\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.981671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.981551    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ad5755-2296-4448-a37c-821c89dc7adc-tmp\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.984007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.983985    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c0ad5755-2296-4448-a37c-821c89dc7adc-tmp\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:51.984135 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:51.984120    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0ad5755-2296-4448-a37c-821c89dc7adc-tls-certs\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:52.005116 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:52.005089    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjkl\" (UniqueName: \"kubernetes.io/projected/c0ad5755-2296-4448-a37c-821c89dc7adc-kube-api-access-bkjkl\") pod \"kube-auth-proxy-66d75c69df-zhqld\" (UID: \"c0ad5755-2296-4448-a37c-821c89dc7adc\") " pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:52.123648 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:52.123564    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"
Sep 17 14:04:52.399111 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:52.399084    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-66d75c69df-zhqld"]
Sep 17 14:04:52.401769 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:04:52.401718    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ad5755_2296_4448_a37c_821c89dc7adc.slice/crio-4d43d11c1f2928fb72b7ef3b4f538a336487fabdf6277dd0df99b352118ab672 WatchSource:0}: Error finding container 4d43d11c1f2928fb72b7ef3b4f538a336487fabdf6277dd0df99b352118ab672: Status 404 returned error can't find the container with id 4d43d11c1f2928fb72b7ef3b4f538a336487fabdf6277dd0df99b352118ab672
Sep 17 14:04:53.083978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.083939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld" event={"ID":"c0ad5755-2296-4448-a37c-821c89dc7adc","Type":"ContainerStarted","Data":"4d43d11c1f2928fb72b7ef3b4f538a336487fabdf6277dd0df99b352118ab672"}
Sep 17 14:04:53.386462 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.386380    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:04:53.390048 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.389916    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.393596 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.393571    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 17 14:04:53.393723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.393613    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 14:04:53.393723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.393661    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-58db9\""
Sep 17 14:04:53.393723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.393577    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 17 14:04:53.393910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.393833    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 17 14:04:53.404255 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.404230    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:04:53.500851 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.500818    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.500874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.500898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.500950    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dj7m\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.500986    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.501030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.501312 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.501075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601659 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601616    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601667    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601718    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dj7m\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601780    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601821    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601863    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.601918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.601899    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.602902 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.602852    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.605618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.605430    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.605762 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.605644    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.605839 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.605813    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.606308 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.606280    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.610922 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.610900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.611023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.610979    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dj7m\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m\") pod \"istiod-openshift-gateway-7b7995d5cd-z7chl\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.703012 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.702973    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:53.991261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:53.991229    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:04:55.107805 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:04:55.107768    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0244d4_48c2_427b_a7bb_782681ba22e1.slice/crio-44817404ad9521459f9c45d5981685405f23e93aecab0025eb3a827e92e6c82e WatchSource:0}: Error finding container 44817404ad9521459f9c45d5981685405f23e93aecab0025eb3a827e92e6c82e: Status 404 returned error can't find the container with id 44817404ad9521459f9c45d5981685405f23e93aecab0025eb3a827e92e6c82e
Sep 17 14:04:56.099941 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:56.099908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld" event={"ID":"c0ad5755-2296-4448-a37c-821c89dc7adc","Type":"ContainerStarted","Data":"8c27873e8d9a3281d863bdd5ab60b75f4dae8c904b77c2ab4388a45cacef120f"}
Sep 17 14:04:56.101580 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:56.101548    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" event={"ID":"7f0244d4-48c2-427b-a7bb-782681ba22e1","Type":"ContainerStarted","Data":"44817404ad9521459f9c45d5981685405f23e93aecab0025eb3a827e92e6c82e"}
Sep 17 14:04:56.120300 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:56.120187    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-66d75c69df-zhqld" podStartSLOduration=2.36412294 podStartE2EDuration="5.120168443s" totalImagesPullingTime="2.756045503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:04:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:04:56.118678127 +0000 UTC m=+538.355555605" watchObservedRunningTime="2026-09-17 14:04:56.120168443 +0000 UTC m=+538.357045921"
Sep 17 14:04:57.748106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:57.748064    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:04:57.748341 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:57.748164    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:04:58.111543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:58.111455    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" event={"ID":"7f0244d4-48c2-427b-a7bb-782681ba22e1","Type":"ContainerStarted","Data":"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"}
Sep 17 14:04:58.111694 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:58.111553    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:04:58.113375 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:58.113346    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-z7chl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 14:04:58.113497 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:58.113393    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:04:58.139610 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:58.139568    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" podStartSLOduration=2.502133791 podStartE2EDuration="5.139552974s" totalImagesPullingTime="2.637419183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:04:58.139238502 +0000 UTC m=+540.376115980" watchObservedRunningTime="2026-09-17 14:04:58.139552974 +0000 UTC m=+540.376430451"
Sep 17 14:04:59.115711 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:59.115681    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-z7chl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 14:04:59.116076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:04:59.115747    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:05:02.116702 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:02.116671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:05:12.438981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.438936    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"]
Sep 17 14:05:12.443969 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.443949    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.447178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.447157    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 17 14:05:12.447404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.447391    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-szj9t\""
Sep 17 14:05:12.463706 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.463684    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"]
Sep 17 14:05:12.475096 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.475078    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"]
Sep 17 14:05:12.484021 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.483998    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.487602 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.487582    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-52tj4\""
Sep 17 14:05:12.489046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.489029    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 17 14:05:12.506026 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.506005    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"]
Sep 17 14:05:12.559245 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.559211    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6b77\" (UniqueName: \"kubernetes.io/projected/95743573-6428-45b9-b3e7-54d7c3ff7382-kube-api-access-k6b77\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.559436 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.559254    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.559436 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.559276    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95743573-6428-45b9-b3e7-54d7c3ff7382-cert\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.559436 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.559360    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcvfc\" (UniqueName: \"kubernetes.io/projected/a9211d7b-c78a-4266-b2b3-7e19b49cf686-kube-api-access-vcvfc\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.660316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.660286    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcvfc\" (UniqueName: \"kubernetes.io/projected/a9211d7b-c78a-4266-b2b3-7e19b49cf686-kube-api-access-vcvfc\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.660489 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.660349    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6b77\" (UniqueName: \"kubernetes.io/projected/95743573-6428-45b9-b3e7-54d7c3ff7382-kube-api-access-k6b77\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.660489 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.660393    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.660489 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.660425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95743573-6428-45b9-b3e7-54d7c3ff7382-cert\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.660601 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:05:12.660526    2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 17 14:05:12.660634 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:05:12.660606    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert podName:a9211d7b-c78a-4266-b2b3-7e19b49cf686 nodeName:}" failed. No retries permitted until 2026-09-17 14:05:13.160589142 +0000 UTC m=+555.397466599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert") pod "llmisvc-controller-manager-777c6bd7d5-xzhzb" (UID: "a9211d7b-c78a-4266-b2b3-7e19b49cf686") : secret "llmisvc-webhook-server-cert" not found
Sep 17 14:05:12.663126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.663107    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95743573-6428-45b9-b3e7-54d7c3ff7382-cert\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.695750 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.695669    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcvfc\" (UniqueName: \"kubernetes.io/projected/a9211d7b-c78a-4266-b2b3-7e19b49cf686-kube-api-access-vcvfc\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:12.695750 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.695700    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6b77\" (UniqueName: \"kubernetes.io/projected/95743573-6428-45b9-b3e7-54d7c3ff7382-kube-api-access-k6b77\") pod \"kserve-controller-manager-d6f4b65b8-wb2tb\" (UID: \"95743573-6428-45b9-b3e7-54d7c3ff7382\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:12.753281 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:12.753240    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:13.029725 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.029700    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"]
Sep 17 14:05:13.032058 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:05:13.032035    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95743573_6428_45b9_b3e7_54d7c3ff7382.slice/crio-3b7948d2e4cbbf262ef99a71dce615742ba4693d1a3b01e5422eb502982053bc WatchSource:0}: Error finding container 3b7948d2e4cbbf262ef99a71dce615742ba4693d1a3b01e5422eb502982053bc: Status 404 returned error can't find the container with id 3b7948d2e4cbbf262ef99a71dce615742ba4693d1a3b01e5422eb502982053bc
Sep 17 14:05:13.164530 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.164481    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:13.167265 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.167234    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb" event={"ID":"95743573-6428-45b9-b3e7-54d7c3ff7382","Type":"ContainerStarted","Data":"3b7948d2e4cbbf262ef99a71dce615742ba4693d1a3b01e5422eb502982053bc"}
Sep 17 14:05:13.167386 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.167371    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9211d7b-c78a-4266-b2b3-7e19b49cf686-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xzhzb\" (UID: \"a9211d7b-c78a-4266-b2b3-7e19b49cf686\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:13.393596 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.393511    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:13.681080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.681051    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"]
Sep 17 14:05:13.684687 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:05:13.684651    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda9211d7b_c78a_4266_b2b3_7e19b49cf686.slice/crio-3b795bbe5887cd2da891faa0033d08d3afc48c2417495a41ddcd662f14a28f20 WatchSource:0}: Error finding container 3b795bbe5887cd2da891faa0033d08d3afc48c2417495a41ddcd662f14a28f20: Status 404 returned error can't find the container with id 3b795bbe5887cd2da891faa0033d08d3afc48c2417495a41ddcd662f14a28f20
Sep 17 14:05:13.910070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.910038    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:05:13.910295 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:13.910258    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery" containerID="cri-o://d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f" gracePeriod=30
Sep 17 14:05:14.151453 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.151432    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:05:14.172496 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.172468    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb" event={"ID":"a9211d7b-c78a-4266-b2b3-7e19b49cf686","Type":"ContainerStarted","Data":"3b795bbe5887cd2da891faa0033d08d3afc48c2417495a41ddcd662f14a28f20"}
Sep 17 14:05:14.173684 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.173657    2570 generic.go:356] "Generic (PLEG): container finished" podID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerID="d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f" exitCode=0
Sep 17 14:05:14.173815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.173703    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"
Sep 17 14:05:14.173815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.173716    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" event={"ID":"7f0244d4-48c2-427b-a7bb-782681ba22e1","Type":"ContainerDied","Data":"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"}
Sep 17 14:05:14.173815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.173776    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" event={"ID":"7f0244d4-48c2-427b-a7bb-782681ba22e1","Type":"ContainerDied","Data":"44817404ad9521459f9c45d5981685405f23e93aecab0025eb3a827e92e6c82e"}
Sep 17 14:05:14.173815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.173803    2570 scope.go:117] "RemoveContainer" containerID="d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"
Sep 17 14:05:14.184005 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.183987    2570 scope.go:117] "RemoveContainer" containerID="d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"
Sep 17 14:05:14.184333 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:05:14.184310    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f\": container with ID starting with d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f not found: ID does not exist" containerID="d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"
Sep 17 14:05:14.184382 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.184346    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f"} err="failed to get container status \"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f\": rpc error: code = NotFound desc = could not find container \"d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f\": container with ID starting with d62a84572a73f72ca50ba801678cf9746d4c75d9be42c8229633bda1bb28d74f not found: ID does not exist"
Sep 17 14:05:14.272635 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.272553    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.272635 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.272599    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.272635 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.272629    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.272939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.272665    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.273042 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.273020    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.273097 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.273074    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.273149 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.273111    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:05:14.273208 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.273167    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dj7m\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m\") pod \"7f0244d4-48c2-427b-a7bb-782681ba22e1\" (UID: \"7f0244d4-48c2-427b-a7bb-782681ba22e1\") "
Sep 17 14:05:14.274076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.273434    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-ca-configmap\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.276038 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.275958    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:05:14.276038 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.275986    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:05:14.276038 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.275993    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs" (OuterVolumeSpecName: "local-certs") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:05:14.276640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.276614    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts" (OuterVolumeSpecName: "cacerts") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:05:14.276762 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.276665    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m" (OuterVolumeSpecName: "kube-api-access-9dj7m") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "kube-api-access-9dj7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:05:14.277069 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.277043    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f0244d4-48c2-427b-a7bb-782681ba22e1" (UID: "7f0244d4-48c2-427b-a7bb-782681ba22e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:05:14.373890 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373845    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dj7m\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-kube-api-access-9dj7m\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.373890 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373893    2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-cacerts\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.374084 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373909    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-csr-dns-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.374084 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373924    2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-kubeconfig\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.374084 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373937    2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f0244d4-48c2-427b-a7bb-782681ba22e1-local-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.374084 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.373950    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f0244d4-48c2-427b-a7bb-782681ba22e1-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:05:14.504095 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.504062    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:05:14.510969 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:14.510943    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl"]
Sep 17 14:05:16.180652 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:16.180617    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" path="/var/lib/kubelet/pods/7f0244d4-48c2-427b-a7bb-782681ba22e1/volumes"
Sep 17 14:05:17.188318 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:17.188287    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb" event={"ID":"95743573-6428-45b9-b3e7-54d7c3ff7382","Type":"ContainerStarted","Data":"39ba3eabea79714203fa6cae7bba861e2360c1cdf8f32643a6d52312382e60c6"}
Sep 17 14:05:17.188658 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:17.188414    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:17.207797 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:17.207754    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb" podStartSLOduration=2.18766137 podStartE2EDuration="5.207718393s" totalImagesPullingTime="3.020057023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:17.207415993 +0000 UTC m=+559.444293472" watchObservedRunningTime="2026-09-17 14:05:17.207718393 +0000 UTC m=+559.444595871"
Sep 17 14:05:18.193443 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:18.193400    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb" event={"ID":"a9211d7b-c78a-4266-b2b3-7e19b49cf686","Type":"ContainerStarted","Data":"12d72bb48c83358606ea25b3d526ab58084cdd4bfad899cb41ae98168099eba1"}
Sep 17 14:05:18.213329 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:18.213284    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb" podStartSLOduration=2.741460347 podStartE2EDuration="6.213268737s" totalImagesPullingTime="3.47180839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:05:18.212847072 +0000 UTC m=+560.449724552" watchObservedRunningTime="2026-09-17 14:05:18.213268737 +0000 UTC m=+560.450146214"
Sep 17 14:05:19.115946 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:19.115909    2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-z7chl container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.19:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 17 14:05:19.116136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:19.115982    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-z7chl" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.19:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 14:05:19.198015 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:19.197982    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:05:48.199001 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:48.198971    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wb2tb"
Sep 17 14:05:50.203608 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:05:50.203572    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xzhzb"
Sep 17 14:06:23.146228 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.146193    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfdzb"]
Sep 17 14:06:23.146697 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.146464    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery"
Sep 17 14:06:23.146697 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.146475    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery"
Sep 17 14:06:23.146697 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.146531    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f0244d4-48c2-427b-a7bb-782681ba22e1" containerName="discovery"
Sep 17 14:06:23.149416 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.149399    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:23.153027 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.153005    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bj58z\""
Sep 17 14:06:23.153136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.153032    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 17 14:06:23.154153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.154136    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 17 14:06:23.165184 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.165157    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfdzb"]
Sep 17 14:06:23.177890 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.177865    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdzqd\" (UniqueName: \"kubernetes.io/projected/b1cc28a7-1085-4d55-abbd-18bcdd569f9c-kube-api-access-wdzqd\") pod \"authorino-operator-76b478bd78-tfdzb\" (UID: \"b1cc28a7-1085-4d55-abbd-18bcdd569f9c\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:23.278534 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.278490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdzqd\" (UniqueName: \"kubernetes.io/projected/b1cc28a7-1085-4d55-abbd-18bcdd569f9c-kube-api-access-wdzqd\") pod \"authorino-operator-76b478bd78-tfdzb\" (UID: \"b1cc28a7-1085-4d55-abbd-18bcdd569f9c\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:23.298630 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.298598    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdzqd\" (UniqueName: \"kubernetes.io/projected/b1cc28a7-1085-4d55-abbd-18bcdd569f9c-kube-api-access-wdzqd\") pod \"authorino-operator-76b478bd78-tfdzb\" (UID: \"b1cc28a7-1085-4d55-abbd-18bcdd569f9c\") " pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:23.459924 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.459891    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:23.741477 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:23.741450    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tfdzb"]
Sep 17 14:06:23.743631 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:23.743594    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1cc28a7_1085_4d55_abbd_18bcdd569f9c.slice/crio-cd36f78f427df39f914f56364011e6983f537f7e8e57a91c1345274bf0cb2311 WatchSource:0}: Error finding container cd36f78f427df39f914f56364011e6983f537f7e8e57a91c1345274bf0cb2311: Status 404 returned error can't find the container with id cd36f78f427df39f914f56364011e6983f537f7e8e57a91c1345274bf0cb2311
Sep 17 14:06:24.445385 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:24.445345    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb" event={"ID":"b1cc28a7-1085-4d55-abbd-18bcdd569f9c","Type":"ContainerStarted","Data":"cd36f78f427df39f914f56364011e6983f537f7e8e57a91c1345274bf0cb2311"}
Sep 17 14:06:25.467229 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.467188    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"]
Sep 17 14:06:25.477552 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.477523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:25.480712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.480672    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 17 14:06:25.480712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.480672    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t2ltw\""
Sep 17 14:06:25.490241 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.490215    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"]
Sep 17 14:06:25.496368 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.496337    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzv97\" (UniqueName: \"kubernetes.io/projected/067db9d3-71d2-4051-ba7f-70a5d6bc96b3-kube-api-access-hzv97\") pod \"dns-operator-controller-manager-7b84cd7c9c-hdgbm\" (UID: \"067db9d3-71d2-4051-ba7f-70a5d6bc96b3\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:25.597798 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.597756    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzv97\" (UniqueName: \"kubernetes.io/projected/067db9d3-71d2-4051-ba7f-70a5d6bc96b3-kube-api-access-hzv97\") pod \"dns-operator-controller-manager-7b84cd7c9c-hdgbm\" (UID: \"067db9d3-71d2-4051-ba7f-70a5d6bc96b3\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:25.609719 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.609676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzv97\" (UniqueName: \"kubernetes.io/projected/067db9d3-71d2-4051-ba7f-70a5d6bc96b3-kube-api-access-hzv97\") pod \"dns-operator-controller-manager-7b84cd7c9c-hdgbm\" (UID: \"067db9d3-71d2-4051-ba7f-70a5d6bc96b3\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:25.790640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:25.790542    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:26.210506 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:26.210413    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"]
Sep 17 14:06:26.213140 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:26.213110    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067db9d3_71d2_4051_ba7f_70a5d6bc96b3.slice/crio-f2306ce6ad55985bfe1ca5e86b9eee2b0f715c8ad8a2b7f2c5563951cabd4e71 WatchSource:0}: Error finding container f2306ce6ad55985bfe1ca5e86b9eee2b0f715c8ad8a2b7f2c5563951cabd4e71: Status 404 returned error can't find the container with id f2306ce6ad55985bfe1ca5e86b9eee2b0f715c8ad8a2b7f2c5563951cabd4e71
Sep 17 14:06:26.455394 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:26.455364    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb" event={"ID":"b1cc28a7-1085-4d55-abbd-18bcdd569f9c","Type":"ContainerStarted","Data":"ed9db103848b8a3e7bce58d1f44b90929ca58b461790f6669f14ef590227bd93"}
Sep 17 14:06:26.455592 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:26.455487    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:26.456898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:26.456875    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm" event={"ID":"067db9d3-71d2-4051-ba7f-70a5d6bc96b3","Type":"ContainerStarted","Data":"f2306ce6ad55985bfe1ca5e86b9eee2b0f715c8ad8a2b7f2c5563951cabd4e71"}
Sep 17 14:06:26.477362 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:26.477305    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb" podStartSLOduration=1.224922225 podStartE2EDuration="3.47728642s" totalImagesPullingTime="2.252364195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:26.476635628 +0000 UTC m=+628.713513098" watchObservedRunningTime="2026-09-17 14:06:26.47728642 +0000 UTC m=+628.714163919"
Sep 17 14:06:28.467027 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:28.466988    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm" event={"ID":"067db9d3-71d2-4051-ba7f-70a5d6bc96b3","Type":"ContainerStarted","Data":"3eef7b9f5e16e329dab262d695d4d7b2cb0dfc066718a89b45015cca31574638"}
Sep 17 14:06:28.467439 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:28.467061    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:28.494352 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:28.494303    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm" podStartSLOduration=1.8776117380000001 podStartE2EDuration="3.494285226s" totalImagesPullingTime="1.616673488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:28.493952054 +0000 UTC m=+630.730829532" watchObservedRunningTime="2026-09-17 14:06:28.494285226 +0000 UTC m=+630.731162702"
Sep 17 14:06:30.210656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.210614    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"]
Sep 17 14:06:30.214073 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.214054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.217089 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.217064    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xbqnv\""
Sep 17 14:06:30.278348 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.278320    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"]
Sep 17 14:06:30.335131 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.335100    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhps\" (UniqueName: \"kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.335325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.335185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.436226 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.436185    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.436226 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.436246    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhps\" (UniqueName: \"kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.436625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.436602    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.446259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.446219    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhps\" (UniqueName: \"kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.524199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.524103    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:30.811854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:30.811769    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"]
Sep 17 14:06:30.815179 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:30.815150    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc290c8b7_22c5_4f3f_8bae_207de82a5459.slice/crio-25045bdf035c3b00714083824f3257404a3f038a5b2e728f05b1c356a9468e63 WatchSource:0}: Error finding container 25045bdf035c3b00714083824f3257404a3f038a5b2e728f05b1c356a9468e63: Status 404 returned error can't find the container with id 25045bdf035c3b00714083824f3257404a3f038a5b2e728f05b1c356a9468e63
Sep 17 14:06:31.482542 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:31.482505    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm" event={"ID":"c290c8b7-22c5-4f3f-8bae-207de82a5459","Type":"ContainerStarted","Data":"25045bdf035c3b00714083824f3257404a3f038a5b2e728f05b1c356a9468e63"}
Sep 17 14:06:32.257331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.257296    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"]
Sep 17 14:06:32.261196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.261168    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:32.264075 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.264048    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tt7rg\""
Sep 17 14:06:32.273654 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.273622    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"]
Sep 17 14:06:32.352412 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.352366    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjt2\" (UniqueName: \"kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2\") pod \"limitador-operator-controller-manager-74495c69f8-fdhjs\" (UID: \"4413ac75-c3d6-48fd-a36a-f02a672aa34f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:32.453605 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.453561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjt2\" (UniqueName: \"kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2\") pod \"limitador-operator-controller-manager-74495c69f8-fdhjs\" (UID: \"4413ac75-c3d6-48fd-a36a-f02a672aa34f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:32.479551 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.479519    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjt2\" (UniqueName: \"kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2\") pod \"limitador-operator-controller-manager-74495c69f8-fdhjs\" (UID: \"4413ac75-c3d6-48fd-a36a-f02a672aa34f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:32.575008 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.574917    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:32.870969 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:32.870880    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"]
Sep 17 14:06:32.875142 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:32.875108    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4413ac75_c3d6_48fd_a36a_f02a672aa34f.slice/crio-1a1608a581d8bb35b12d513a15768f8360a2129141812c1cdf02c5811770e2a3 WatchSource:0}: Error finding container 1a1608a581d8bb35b12d513a15768f8360a2129141812c1cdf02c5811770e2a3: Status 404 returned error can't find the container with id 1a1608a581d8bb35b12d513a15768f8360a2129141812c1cdf02c5811770e2a3
Sep 17 14:06:33.494424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:33.494381    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs" event={"ID":"4413ac75-c3d6-48fd-a36a-f02a672aa34f","Type":"ContainerStarted","Data":"1a1608a581d8bb35b12d513a15768f8360a2129141812c1cdf02c5811770e2a3"}
Sep 17 14:06:34.503373 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:34.503325    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm" event={"ID":"c290c8b7-22c5-4f3f-8bae-207de82a5459","Type":"ContainerStarted","Data":"a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885"}
Sep 17 14:06:34.503842 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:34.503552    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:34.536930 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:34.536865    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm" podStartSLOduration=1.055250541 podStartE2EDuration="4.536842191s" totalImagesPullingTime="3.48159165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:34.534163768 +0000 UTC m=+636.771041245" watchObservedRunningTime="2026-09-17 14:06:34.536842191 +0000 UTC m=+636.773719671"
Sep 17 14:06:35.510438 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:35.510398    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs" event={"ID":"4413ac75-c3d6-48fd-a36a-f02a672aa34f","Type":"ContainerStarted","Data":"e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8"}
Sep 17 14:06:35.510831 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:35.510624    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:35.536648 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:35.536592    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs" podStartSLOduration=1.3773457869999999 podStartE2EDuration="3.536570885s" totalImagesPullingTime="2.159225098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:35.535452437 +0000 UTC m=+637.772329916" watchObservedRunningTime="2026-09-17 14:06:35.536570885 +0000 UTC m=+637.773448362"
Sep 17 14:06:37.312798 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.312764    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"]
Sep 17 14:06:37.315936 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.315919    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.318801 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.318718    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 17 14:06:37.318801 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.318789    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 17 14:06:37.319036 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.318764    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mjrpk\""
Sep 17 14:06:37.326907 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.326878    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"]
Sep 17 14:06:37.401409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.401374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.401578 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.401432    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.401578 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.401502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhp6p\" (UniqueName: \"kubernetes.io/projected/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-kube-api-access-qhp6p\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.463295 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.463268    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-tfdzb"
Sep 17 14:06:37.501951 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.501917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.502130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.501976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhp6p\" (UniqueName: \"kubernetes.io/projected/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-kube-api-access-qhp6p\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.502130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.502005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.502711 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.502679    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.505040 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.505009    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.511604 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.511573    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhp6p\" (UniqueName: \"kubernetes.io/projected/22b2221d-1485-40f0-bb8f-3ca3f1dd0dae-kube-api-access-qhp6p\") pod \"kuadrant-console-plugin-fbdd4f445-pqrlh\" (UID: \"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.626580 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.626481    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"
Sep 17 14:06:37.903594 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:37.903567    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh"]
Sep 17 14:06:37.906282 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:37.906253    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22b2221d_1485_40f0_bb8f_3ca3f1dd0dae.slice/crio-68ffbadc2881f69b862cf03ada7359476c580e4a39476c066e760a5b6a2eb7e0 WatchSource:0}: Error finding container 68ffbadc2881f69b862cf03ada7359476c580e4a39476c066e760a5b6a2eb7e0: Status 404 returned error can't find the container with id 68ffbadc2881f69b862cf03ada7359476c580e4a39476c066e760a5b6a2eb7e0
Sep 17 14:06:38.525348 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:38.525313    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh" event={"ID":"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae","Type":"ContainerStarted","Data":"68ffbadc2881f69b862cf03ada7359476c580e4a39476c066e760a5b6a2eb7e0"}
Sep 17 14:06:39.474838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:39.474795    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hdgbm"
Sep 17 14:06:43.553843 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:43.553802    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh" event={"ID":"22b2221d-1485-40f0-bb8f-3ca3f1dd0dae","Type":"ContainerStarted","Data":"2b7b6d9ac9f291b412bbc4c44ef850c6f3b9f21a46048b326508c41568f49451"}
Sep 17 14:06:43.574819 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:43.574761    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-pqrlh" podStartSLOduration=1.8659602180000001 podStartE2EDuration="6.574727485s" totalImagesPullingTime="4.708767267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:43.572935001 +0000 UTC m=+645.809812479" watchObservedRunningTime="2026-09-17 14:06:43.574727485 +0000 UTC m=+645.811604959"
Sep 17 14:06:45.513487 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:45.513455    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:46.518123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:46.518091    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.601171 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.601121    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"]
Sep 17 14:06:47.601807 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.601327    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" containerName="manager" containerID="cri-o://a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885" gracePeriod=2
Sep 17 14:06:47.603704 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.603676    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"]
Sep 17 14:06:47.603952 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.603918    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.618584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.618548    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"]
Sep 17 14:06:47.619411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.618857    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" containerName="manager" containerID="cri-o://e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8" gracePeriod=2
Sep 17 14:06:47.620878 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.620850    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"]
Sep 17 14:06:47.621504 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.621462    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.623694 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.623662    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.632469 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.632382    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:06:47.632905 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.632886    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" containerName="manager"
Sep 17 14:06:47.632960 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.632911    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" containerName="manager"
Sep 17 14:06:47.632960 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.632935    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" containerName="manager"
Sep 17 14:06:47.632960 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.632944    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" containerName="manager"
Sep 17 14:06:47.633052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.633031    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" containerName="manager"
Sep 17 14:06:47.633052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.633045    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" containerName="manager"
Sep 17 14:06:47.636153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.636134    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.645132 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.645096    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"]
Sep 17 14:06:47.648971 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.648928    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:47.651131 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.651097    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:06:47.658974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.658944    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"]
Sep 17 14:06:47.687456 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.687420    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.689507 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.689482    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.691452 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.691420    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.693565 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.693539    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.765679 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.765635    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:06:47.769513 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.769490    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.772160 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.772126    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.785365 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.785329    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:06:47.793764 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.793697    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt6dn\" (UniqueName: \"kubernetes.io/projected/97ed6e53-800e-448a-87bd-d0a65f4d075b-kube-api-access-kt6dn\") pod \"limitador-operator-controller-manager-74495c69f8-wnzng\" (UID: \"97ed6e53-800e-448a-87bd-d0a65f4d075b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:47.793764 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.793772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b58tq\" (UniqueName: \"kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.794001 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.793804    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.802536 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.802494    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.883413 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.883388    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.885838 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.885802    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.886067 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.886051    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.888122 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.888097    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.890228 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.890201    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:47.892133 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:47.892107    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:47.894486 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.894460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.894559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.894507    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt6dn\" (UniqueName: \"kubernetes.io/projected/97ed6e53-800e-448a-87bd-d0a65f4d075b-kube-api-access-kt6dn\") pod \"limitador-operator-controller-manager-74495c69f8-wnzng\" (UID: \"97ed6e53-800e-448a-87bd-d0a65f4d075b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:47.894559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.894535    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgd68\" (UniqueName: \"kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.894670 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.894586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b58tq\" (UniqueName: \"kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.894670 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.894646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.895047 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.895026    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.903105 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.903075    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b58tq\" (UniqueName: \"kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-d6j7q\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:47.903105 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.903102    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt6dn\" (UniqueName: \"kubernetes.io/projected/97ed6e53-800e-448a-87bd-d0a65f4d075b-kube-api-access-kt6dn\") pod \"limitador-operator-controller-manager-74495c69f8-wnzng\" (UID: \"97ed6e53-800e-448a-87bd-d0a65f4d075b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:47.996000 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.995961    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume\") pod \"c290c8b7-22c5-4f3f-8bae-207de82a5459\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") "
Sep 17 14:06:47.996183 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996017    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpjt2\" (UniqueName: \"kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2\") pod \"4413ac75-c3d6-48fd-a36a-f02a672aa34f\" (UID: \"4413ac75-c3d6-48fd-a36a-f02a672aa34f\") "
Sep 17 14:06:47.996183 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996052    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hhps\" (UniqueName: \"kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps\") pod \"c290c8b7-22c5-4f3f-8bae-207de82a5459\" (UID: \"c290c8b7-22c5-4f3f-8bae-207de82a5459\") "
Sep 17 14:06:47.996298 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996183    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.996298 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996244    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgd68\" (UniqueName: \"kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.996583 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996556    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:47.996708 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.996609    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c290c8b7-22c5-4f3f-8bae-207de82a5459" (UID: "c290c8b7-22c5-4f3f-8bae-207de82a5459"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:06:47.998460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.998431    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps" (OuterVolumeSpecName: "kube-api-access-7hhps") pod "c290c8b7-22c5-4f3f-8bae-207de82a5459" (UID: "c290c8b7-22c5-4f3f-8bae-207de82a5459"). InnerVolumeSpecName "kube-api-access-7hhps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:06:47.998460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:47.998437    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2" (OuterVolumeSpecName: "kube-api-access-rpjt2") pod "4413ac75-c3d6-48fd-a36a-f02a672aa34f" (UID: "4413ac75-c3d6-48fd-a36a-f02a672aa34f"). InnerVolumeSpecName "kube-api-access-rpjt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:06:48.008408 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.008380    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgd68\" (UniqueName: \"kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68\") pod \"kuadrant-operator-controller-manager-5769f99d9f-2hxc9\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:48.025838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.025792    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:48.031975 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.031943    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:48.081013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.080977    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:48.097981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.097902    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c290c8b7-22c5-4f3f-8bae-207de82a5459-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:06:48.097981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.097936    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpjt2\" (UniqueName: \"kubernetes.io/projected/4413ac75-c3d6-48fd-a36a-f02a672aa34f-kube-api-access-rpjt2\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:06:48.097981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.097954    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hhps\" (UniqueName: \"kubernetes.io/projected/c290c8b7-22c5-4f3f-8bae-207de82a5459-kube-api-access-7hhps\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:06:48.184312 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:48.181990    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-fdhjs\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:48.184312 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:48.184210    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-s2bhm\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:48.186707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.186668    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" path="/var/lib/kubelet/pods/4413ac75-c3d6-48fd-a36a-f02a672aa34f/volumes"
Sep 17 14:06:48.187773 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.187713    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c290c8b7-22c5-4f3f-8bae-207de82a5459" path="/var/lib/kubelet/pods/c290c8b7-22c5-4f3f-8bae-207de82a5459/volumes"
Sep 17 14:06:48.311033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.311008    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"]
Sep 17 14:06:48.314031 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:48.313990    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ed6e53_800e_448a_87bd_d0a65f4d075b.slice/crio-b26b0303dd76ceb1cb06a8eebfcdcaaf9fce96c6a4b70d612f36a05b11851b93 WatchSource:0}: Error finding container b26b0303dd76ceb1cb06a8eebfcdcaaf9fce96c6a4b70d612f36a05b11851b93: Status 404 returned error can't find the container with id b26b0303dd76ceb1cb06a8eebfcdcaaf9fce96c6a4b70d612f36a05b11851b93
Sep 17 14:06:48.314458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.314435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:06:48.318183 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:48.318153    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73ea6a3_62c1_40fc_a66e_e79e3ae7deb5.slice/crio-810b3df433d5da901fdfc165f8c97879c52b8caf0c0a8ff3999ddc7ea44e3644 WatchSource:0}: Error finding container 810b3df433d5da901fdfc165f8c97879c52b8caf0c0a8ff3999ddc7ea44e3644: Status 404 returned error can't find the container with id 810b3df433d5da901fdfc165f8c97879c52b8caf0c0a8ff3999ddc7ea44e3644
Sep 17 14:06:48.372972 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.372721    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:06:48.375877 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:06:48.375850    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3c4293_8e10_45da_92e3_bbeb21be55d5.slice/crio-85a9bee9396f9bd430f4c32592677852bc202a25c3f9b1e24c6480eea8304100 WatchSource:0}: Error finding container 85a9bee9396f9bd430f4c32592677852bc202a25c3f9b1e24c6480eea8304100: Status 404 returned error can't find the container with id 85a9bee9396f9bd430f4c32592677852bc202a25c3f9b1e24c6480eea8304100
Sep 17 14:06:48.581006 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.580970    2570 generic.go:356] "Generic (PLEG): container finished" podID="4413ac75-c3d6-48fd-a36a-f02a672aa34f" containerID="e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8" exitCode=0
Sep 17 14:06:48.581223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.581036    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-fdhjs"
Sep 17 14:06:48.581223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.581097    2570 scope.go:117] "RemoveContainer" containerID="e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8"
Sep 17 14:06:48.583546 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.583515    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng" event={"ID":"97ed6e53-800e-448a-87bd-d0a65f4d075b","Type":"ContainerStarted","Data":"4f5b0e6674608f65d4ce5c03c5c2c7ee503945a2d62be0855af48c3462b630e6"}
Sep 17 14:06:48.583775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.583589    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng" event={"ID":"97ed6e53-800e-448a-87bd-d0a65f4d075b","Type":"ContainerStarted","Data":"b26b0303dd76ceb1cb06a8eebfcdcaaf9fce96c6a4b70d612f36a05b11851b93"}
Sep 17 14:06:48.583775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.583668    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:48.584974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.584952    2570 generic.go:356] "Generic (PLEG): container finished" podID="c290c8b7-22c5-4f3f-8bae-207de82a5459" containerID="a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885" exitCode=0
Sep 17 14:06:48.585068 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.585012    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s2bhm"
Sep 17 14:06:48.586920 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.586895    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" event={"ID":"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5","Type":"ContainerStarted","Data":"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"}
Sep 17 14:06:48.587034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.586937    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" event={"ID":"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5","Type":"ContainerStarted","Data":"810b3df433d5da901fdfc165f8c97879c52b8caf0c0a8ff3999ddc7ea44e3644"}
Sep 17 14:06:48.587034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.586960    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:48.588921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.588901    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerStarted","Data":"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"}
Sep 17 14:06:48.589015 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.588938    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerStarted","Data":"85a9bee9396f9bd430f4c32592677852bc202a25c3f9b1e24c6480eea8304100"}
Sep 17 14:06:48.589059 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.589018    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:48.591491 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.591473    2570 scope.go:117] "RemoveContainer" containerID="e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8"
Sep 17 14:06:48.591814 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:48.591785    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8\": container with ID starting with e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8 not found: ID does not exist" containerID="e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8"
Sep 17 14:06:48.591901 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.591825    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8"} err="failed to get container status \"e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8\": rpc error: code = NotFound desc = could not find container \"e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8\": container with ID starting with e1ee00a1ff06b09f6f6d17429ff08e1700164b5ec9fc75cc2a7f8b57ee9e62f8 not found: ID does not exist"
Sep 17 14:06:48.591901 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.591848    2570 scope.go:117] "RemoveContainer" containerID="a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885"
Sep 17 14:06:48.600193 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.600175    2570 scope.go:117] "RemoveContainer" containerID="a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885"
Sep 17 14:06:48.600555 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:06:48.600517    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885\": container with ID starting with a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885 not found: ID does not exist" containerID="a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885"
Sep 17 14:06:48.600624 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.600564    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885"} err="failed to get container status \"a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885\": rpc error: code = NotFound desc = could not find container \"a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885\": container with ID starting with a9de040ebdca0e6b1bd7a57f086bb6cb88586b9953f95c7adb200b68eda7d885 not found: ID does not exist"
Sep 17 14:06:48.609837 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.609787    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng" podStartSLOduration=1.609766501 podStartE2EDuration="1.609766501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:48.608292401 +0000 UTC m=+650.845169878" watchObservedRunningTime="2026-09-17 14:06:48.609766501 +0000 UTC m=+650.846643974"
Sep 17 14:06:48.630894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.630837    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" podStartSLOduration=1.630813167 podStartE2EDuration="1.630813167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:48.630341908 +0000 UTC m=+650.867219385" watchObservedRunningTime="2026-09-17 14:06:48.630813167 +0000 UTC m=+650.867690644"
Sep 17 14:06:48.653136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:48.653072    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" podStartSLOduration=1.6530472170000001 podStartE2EDuration="1.653047217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:06:48.652527249 +0000 UTC m=+650.889404749" watchObservedRunningTime="2026-09-17 14:06:48.653047217 +0000 UTC m=+650.889924696"
Sep 17 14:06:59.599363 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.599330    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:06:59.599827 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.599407    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wnzng"
Sep 17 14:06:59.599827 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.599440    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:06:59.723817 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.723786    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:06:59.724010 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.723988    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" podUID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" containerName="manager" containerID="cri-o://f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed" gracePeriod=10
Sep 17 14:06:59.966199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:06:59.966175    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:07:00.098251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.098224    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume\") pod \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") "
Sep 17 14:07:00.098251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.098256    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b58tq\" (UniqueName: \"kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq\") pod \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\" (UID: \"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5\") "
Sep 17 14:07:00.098638 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.098615    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" (UID: "f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:07:00.100299 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.100258    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq" (OuterVolumeSpecName: "kube-api-access-b58tq") pod "f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" (UID: "f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5"). InnerVolumeSpecName "kube-api-access-b58tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:07:00.198763 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.198720    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:07:00.198763 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.198760    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b58tq\" (UniqueName: \"kubernetes.io/projected/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5-kube-api-access-b58tq\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:07:00.652318 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.652286    2570 generic.go:356] "Generic (PLEG): container finished" podID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" containerID="f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed" exitCode=0
Sep 17 14:07:00.652689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.652357    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" event={"ID":"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5","Type":"ContainerDied","Data":"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"}
Sep 17 14:07:00.652689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.652405    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q" event={"ID":"f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5","Type":"ContainerDied","Data":"810b3df433d5da901fdfc165f8c97879c52b8caf0c0a8ff3999ddc7ea44e3644"}
Sep 17 14:07:00.652689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.652342    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"
Sep 17 14:07:00.652689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.652426    2570 scope.go:117] "RemoveContainer" containerID="f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"
Sep 17 14:07:00.660504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.660488    2570 scope.go:117] "RemoveContainer" containerID="f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"
Sep 17 14:07:00.660806 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:07:00.660787    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed\": container with ID starting with f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed not found: ID does not exist" containerID="f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"
Sep 17 14:07:00.660862 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.660815    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed"} err="failed to get container status \"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed\": rpc error: code = NotFound desc = could not find container \"f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed\": container with ID starting with f2f4ed4f1d4c159bc55355f33bc2e04994d28de92c16966d0633e8c4529826ed not found: ID does not exist"
Sep 17 14:07:00.682607 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.682586    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:07:00.692052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:00.692032    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-d6j7q"]
Sep 17 14:07:02.181459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:02.181426    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" path="/var/lib/kubelet/pods/f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5/volumes"
Sep 17 14:07:16.088465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.088422    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"]
Sep 17 14:07:16.088944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.088931    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" containerName="manager"
Sep 17 14:07:16.089022 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.088951    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" containerName="manager"
Sep 17 14:07:16.089126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.089110    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73ea6a3-62c1-40fc-a66e-e79e3ae7deb5" containerName="manager"
Sep 17 14:07:16.107640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.107601    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"]
Sep 17 14:07:16.107835 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.107754    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.110922 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.110885    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-p7ghv\""
Sep 17 14:07:16.110922 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.110882    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 17 14:07:16.117422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117398    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117567 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117430    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957003c4-9cea-44ba-a8db-af7c2caf48b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117567 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117567 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117512    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117567 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117556    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117727 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117604    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117727 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76wmn\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-kube-api-access-76wmn\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117727 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117639    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.117727 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.117662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218260 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218290    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218309    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957003c4-9cea-44ba-a8db-af7c2caf48b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218394    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218467    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218495 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218492    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76wmn\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-kube-api-access-76wmn\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218940 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.218940 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.218716    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.219130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.219025    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.219195 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.219167    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/957003c4-9cea-44ba-a8db-af7c2caf48b6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.219271 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.219253    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.219330 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.219288    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.221210 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.221188    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.221325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.221241    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.227369 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.227338    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.227499 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.227444    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76wmn\" (UniqueName: \"kubernetes.io/projected/957003c4-9cea-44ba-a8db-af7c2caf48b6-kube-api-access-76wmn\") pod \"maas-default-gateway-openshift-default-5ccb968c66-lk9v4\" (UID: \"957003c4-9cea-44ba-a8db-af7c2caf48b6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.419199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.419160    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:16.696987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.696959    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"]
Sep 17 14:07:16.699215 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:07:16.699186    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957003c4_9cea_44ba_a8db_af7c2caf48b6.slice/crio-2b4e74787610404efe41be58fd20d4ad329171da435eda35fe6b580691b9976e WatchSource:0}: Error finding container 2b4e74787610404efe41be58fd20d4ad329171da435eda35fe6b580691b9976e: Status 404 returned error can't find the container with id 2b4e74787610404efe41be58fd20d4ad329171da435eda35fe6b580691b9976e
Sep 17 14:07:16.727135 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:16.727092    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" event={"ID":"957003c4-9cea-44ba-a8db-af7c2caf48b6","Type":"ContainerStarted","Data":"2b4e74787610404efe41be58fd20d4ad329171da435eda35fe6b580691b9976e"}
Sep 17 14:07:19.527244 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:19.527203    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:07:19.527526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:19.527291    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:07:19.527526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:19.527359    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:07:19.751337 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:19.751293    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" event={"ID":"957003c4-9cea-44ba-a8db-af7c2caf48b6","Type":"ContainerStarted","Data":"9bc39b4ca05d9c0bee7420fd33f547c0aed443f04c7a8c7facfef5db8ee1a402"}
Sep 17 14:07:19.774118 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:19.774062    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podStartSLOduration=0.948453692 podStartE2EDuration="3.774043272s" totalImagesPullingTime="2.82558958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:07:19.772695624 +0000 UTC m=+682.009573104" watchObservedRunningTime="2026-09-17 14:07:19.774043272 +0000 UTC m=+682.010920748"
Sep 17 14:07:20.419640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:20.419596    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:20.424256 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:20.424233    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:20.756879 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:20.756779    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:20.757848 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:20.757827    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:07:21.127069 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.126985    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:07:21.133769 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.133707    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.136700 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.136673    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 17 14:07:21.143198 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.143174    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:07:21.160427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.160392    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g97hq\" (UniqueName: \"kubernetes.io/projected/14547245-c2cf-403b-9e62-11154b55a41f-kube-api-access-g97hq\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.160599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.160457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/14547245-c2cf-403b-9e62-11154b55a41f-config-file\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.171310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.171279    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:07:21.260921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.260879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/14547245-c2cf-403b-9e62-11154b55a41f-config-file\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.261128 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.260991    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g97hq\" (UniqueName: \"kubernetes.io/projected/14547245-c2cf-403b-9e62-11154b55a41f-kube-api-access-g97hq\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.261571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.261546    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/14547245-c2cf-403b-9e62-11154b55a41f-config-file\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.283503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.283474    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g97hq\" (UniqueName: \"kubernetes.io/projected/14547245-c2cf-403b-9e62-11154b55a41f-kube-api-access-g97hq\") pod \"limitador-limitador-84994f7ffc-lkk9s\" (UID: \"14547245-c2cf-403b-9e62-11154b55a41f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.445839 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.445796    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:21.730687 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.730658    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:07:21.732846 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:07:21.732789    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14547245_c2cf_403b_9e62_11154b55a41f.slice/crio-192c44f53bc85cc3c2638f574dd74022d804203932ceae5bdebf8d48d648b158 WatchSource:0}: Error finding container 192c44f53bc85cc3c2638f574dd74022d804203932ceae5bdebf8d48d648b158: Status 404 returned error can't find the container with id 192c44f53bc85cc3c2638f574dd74022d804203932ceae5bdebf8d48d648b158
Sep 17 14:07:21.762014 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.761973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s" event={"ID":"14547245-c2cf-403b-9e62-11154b55a41f","Type":"ContainerStarted","Data":"192c44f53bc85cc3c2638f574dd74022d804203932ceae5bdebf8d48d648b158"}
Sep 17 14:07:21.839199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.839169    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:21.841632 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.841617    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:21.844910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.844890    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-tsglc\""
Sep 17 14:07:21.850140 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.850118    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:21.865293 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.865270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p677d\" (UniqueName: \"kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d\") pod \"authorino-59c974bd48-chcb2\" (UID: \"816ef189-0b6c-468a-89d9-596143061d8d\") " pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:21.966307 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.966267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p677d\" (UniqueName: \"kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d\") pod \"authorino-59c974bd48-chcb2\" (UID: \"816ef189-0b6c-468a-89d9-596143061d8d\") " pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:21.975816 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:21.975791    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p677d\" (UniqueName: \"kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d\") pod \"authorino-59c974bd48-chcb2\" (UID: \"816ef189-0b6c-468a-89d9-596143061d8d\") " pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:22.152395 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:22.152361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:22.427765 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:22.427725    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:22.429919 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:07:22.429891    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod816ef189_0b6c_468a_89d9_596143061d8d.slice/crio-09a0d0926025aa3f59f29bc9aa63f00fb3f8aa4b2529ed4b2e3f3dee21c0b444 WatchSource:0}: Error finding container 09a0d0926025aa3f59f29bc9aa63f00fb3f8aa4b2529ed4b2e3f3dee21c0b444: Status 404 returned error can't find the container with id 09a0d0926025aa3f59f29bc9aa63f00fb3f8aa4b2529ed4b2e3f3dee21c0b444
Sep 17 14:07:22.767983 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:22.767890    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-chcb2" event={"ID":"816ef189-0b6c-468a-89d9-596143061d8d","Type":"ContainerStarted","Data":"09a0d0926025aa3f59f29bc9aa63f00fb3f8aa4b2529ed4b2e3f3dee21c0b444"}
Sep 17 14:07:24.778138 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:24.778100    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s" event={"ID":"14547245-c2cf-403b-9e62-11154b55a41f","Type":"ContainerStarted","Data":"c1114621ee9fa704f9f10fe86f404995c49d01915138c167feecc8bd88bcd7f7"}
Sep 17 14:07:24.778532 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:24.778203    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:24.798882 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:24.798836    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s" podStartSLOduration=1.6908663910000001 podStartE2EDuration="3.798816302s" totalImagesPullingTime="2.107949911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:07:24.796281846 +0000 UTC m=+687.033159324" watchObservedRunningTime="2026-09-17 14:07:24.798816302 +0000 UTC m=+687.035693779"
Sep 17 14:07:25.785317 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:25.785272    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-chcb2" event={"ID":"816ef189-0b6c-468a-89d9-596143061d8d","Type":"ContainerStarted","Data":"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"}
Sep 17 14:07:25.805308 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:25.805255    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-chcb2" podStartSLOduration=2.454237862 podStartE2EDuration="4.80523716s" totalImagesPullingTime="2.350999298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:07:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:07:25.804790833 +0000 UTC m=+688.041668310" watchObservedRunningTime="2026-09-17 14:07:25.80523716 +0000 UTC m=+688.042114637"
Sep 17 14:07:35.787089 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:35.787052    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"
Sep 17 14:07:57.077124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.077032    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:57.077558 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.077253    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-chcb2" podUID="816ef189-0b6c-468a-89d9-596143061d8d" containerName="authorino" containerID="cri-o://ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369" gracePeriod=30
Sep 17 14:07:57.165532 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.165496    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:07:57.167878 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.167863    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.170774 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.170755    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 17 14:07:57.178198 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.178171    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:07:57.280378 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.280335    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.280566 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.280395    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbvgx\" (UniqueName: \"kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.341974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.341905    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:57.381305 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.381267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbvgx\" (UniqueName: \"kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.381484 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.381413    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.384047 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.384021    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.390426 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.390399    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbvgx\" (UniqueName: \"kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx\") pod \"authorino-5ccdbd7b7-tj4nd\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") " pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.477689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.477656    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:07:57.482663 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.482644    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p677d\" (UniqueName: \"kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d\") pod \"816ef189-0b6c-468a-89d9-596143061d8d\" (UID: \"816ef189-0b6c-468a-89d9-596143061d8d\") "
Sep 17 14:07:57.484800 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.484772    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d" (OuterVolumeSpecName: "kube-api-access-p677d") pod "816ef189-0b6c-468a-89d9-596143061d8d" (UID: "816ef189-0b6c-468a-89d9-596143061d8d"). InnerVolumeSpecName "kube-api-access-p677d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:07:57.583676 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.583633    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p677d\" (UniqueName: \"kubernetes.io/projected/816ef189-0b6c-468a-89d9-596143061d8d-kube-api-access-p677d\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:07:57.750902 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.750857    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:07:57.753092 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:07:57.753069    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45b7b6fb_dd8d_44c8_abd4_c8a347bc6bcc.slice/crio-7593190a3fd9bc013faee2bd7d6d5c0eb82fca4cbff2b97934727796763b59b6 WatchSource:0}: Error finding container 7593190a3fd9bc013faee2bd7d6d5c0eb82fca4cbff2b97934727796763b59b6: Status 404 returned error can't find the container with id 7593190a3fd9bc013faee2bd7d6d5c0eb82fca4cbff2b97934727796763b59b6
Sep 17 14:07:57.936613 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.936581    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" event={"ID":"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc","Type":"ContainerStarted","Data":"7593190a3fd9bc013faee2bd7d6d5c0eb82fca4cbff2b97934727796763b59b6"}
Sep 17 14:07:57.937881 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.937858    2570 generic.go:356] "Generic (PLEG): container finished" podID="816ef189-0b6c-468a-89d9-596143061d8d" containerID="ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369" exitCode=0
Sep 17 14:07:57.937997 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.937903    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-chcb2"
Sep 17 14:07:57.937997 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.937946    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-chcb2" event={"ID":"816ef189-0b6c-468a-89d9-596143061d8d","Type":"ContainerDied","Data":"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"}
Sep 17 14:07:57.937997 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.937983    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-chcb2" event={"ID":"816ef189-0b6c-468a-89d9-596143061d8d","Type":"ContainerDied","Data":"09a0d0926025aa3f59f29bc9aa63f00fb3f8aa4b2529ed4b2e3f3dee21c0b444"}
Sep 17 14:07:57.938100 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.938000    2570 scope.go:117] "RemoveContainer" containerID="ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"
Sep 17 14:07:57.946724 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.946704    2570 scope.go:117] "RemoveContainer" containerID="ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"
Sep 17 14:07:57.947100 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:07:57.947079    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369\": container with ID starting with ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369 not found: ID does not exist" containerID="ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"
Sep 17 14:07:57.947176 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.947110    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369"} err="failed to get container status \"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369\": rpc error: code = NotFound desc = could not find container \"ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369\": container with ID starting with ac042f36e3aa699ec683f815e3d518ec9dd64fa30cbb5da44e138dfdae88f369 not found: ID does not exist"
Sep 17 14:07:57.962059 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.962025    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:57.966480 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:57.966450    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-chcb2"]
Sep 17 14:07:58.185540 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:58.182321    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="816ef189-0b6c-468a-89d9-596143061d8d" path="/var/lib/kubelet/pods/816ef189-0b6c-468a-89d9-596143061d8d/volumes"
Sep 17 14:07:58.944124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:58.944084    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" event={"ID":"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc","Type":"ContainerStarted","Data":"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"}
Sep 17 14:07:58.961177 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:07:58.961127    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" podStartSLOduration=1.450504878 podStartE2EDuration="1.96111097s" totalImagesPullingTime="510.606092ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:07:58.960553166 +0000 UTC m=+721.197430624" watchObservedRunningTime="2026-09-17 14:07:58.96111097 +0000 UTC m=+721.197988447"
Sep 17 14:08:39.268237 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:39.268185    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:08:49.171891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.171853    2570 generic.go:356] "Generic (PLEG): container finished" podID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerID="9bc39b4ca05d9c0bee7420fd33f547c0aed443f04c7a8c7facfef5db8ee1a402" exitCode=0
Sep 17 14:08:49.172333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.171930    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" event={"ID":"957003c4-9cea-44ba-a8db-af7c2caf48b6","Type":"ContainerDied","Data":"9bc39b4ca05d9c0bee7420fd33f547c0aed443f04c7a8c7facfef5db8ee1a402"}
Sep 17 14:08:49.172333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.172305    2570 scope.go:117] "RemoveContainer" containerID="9bc39b4ca05d9c0bee7420fd33f547c0aed443f04c7a8c7facfef5db8ee1a402"
Sep 17 14:08:49.173371 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.173341    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:08:49.173484 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.173430    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:08:49.173484 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:49.173477    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:08:50.182603 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:50.182570    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" event={"ID":"957003c4-9cea-44ba-a8db-af7c2caf48b6","Type":"ContainerStarted","Data":"8b39b73b30dc827171156f1c73eb368920ed2bb430e2a6aa2ba1810370d1afad"}
Sep 17 14:08:50.419940 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:50.419905    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:08:50.421192 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:50.421164    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lk9v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 17 14:08:50.421317 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:50.421218    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podUID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 17 14:08:50.757680 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:50.757638    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:08:51.420207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:51.420167    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lk9v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 17 14:08:51.420632 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:51.420245    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podUID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 17 14:08:52.419695 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:52.419659    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lk9v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 17 14:08:52.419915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:52.419710    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podUID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 17 14:08:53.420480 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:53.420445    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lk9v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 17 14:08:53.420887 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:53.420516    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podUID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 17 14:08:54.420386 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:54.420347    2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-lk9v4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 17 14:08:54.420590 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:54.420403    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4" podUID="957003c4-9cea-44ba-a8db-af7c2caf48b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 17 14:08:55.424561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:55.424521    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:08:55.425720 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:08:55.425697    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-lk9v4"
Sep 17 14:09:01.323976 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:01.323942    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:09:08.411009 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:08.410979    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:09:14.910645 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.910615    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"]
Sep 17 14:09:14.910996 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.910959    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="816ef189-0b6c-468a-89d9-596143061d8d" containerName="authorino"
Sep 17 14:09:14.910996 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.910970    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="816ef189-0b6c-468a-89d9-596143061d8d" containerName="authorino"
Sep 17 14:09:14.911075 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.911036    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="816ef189-0b6c-468a-89d9-596143061d8d" containerName="authorino"
Sep 17 14:09:14.913343 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.913328    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:14.916236 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.916213    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 17 14:09:14.917357 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.917333    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 17 14:09:14.917357 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.917352    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 17 14:09:14.917569 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.917342    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-98swh\""
Sep 17 14:09:14.926516 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:14.926490    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"]
Sep 17 14:09:15.051748 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.051909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051805    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.051909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.051909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.051909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051866    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq829\" (UniqueName: \"kubernetes.io/projected/830200e7-15e8-4dba-99be-0dd1cacf8f29-kube-api-access-fq829\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.051909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.051895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830200e7-15e8-4dba-99be-0dd1cacf8f29-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152619 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152581    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152619 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152634    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq829\" (UniqueName: \"kubernetes.io/projected/830200e7-15e8-4dba-99be-0dd1cacf8f29-kube-api-access-fq829\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152718    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830200e7-15e8-4dba-99be-0dd1cacf8f29-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.152894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.152783    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.153170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.153150    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.153582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.153555    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.153716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.153631    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.155095 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.155074    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/830200e7-15e8-4dba-99be-0dd1cacf8f29-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.155429 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.155409    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/830200e7-15e8-4dba-99be-0dd1cacf8f29-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.163948 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.163887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq829\" (UniqueName: \"kubernetes.io/projected/830200e7-15e8-4dba-99be-0dd1cacf8f29-kube-api-access-fq829\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz\" (UID: \"830200e7-15e8-4dba-99be-0dd1cacf8f29\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.223888 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.223861    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:15.502000 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.501980    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"]
Sep 17 14:09:15.504269 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:09:15.504229    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod830200e7_15e8_4dba_99be_0dd1cacf8f29.slice/crio-751bf5d185e83ddaedb0d4433d917c9b9ba1822311f135611b8c90ffd8bca42b WatchSource:0}: Error finding container 751bf5d185e83ddaedb0d4433d917c9b9ba1822311f135611b8c90ffd8bca42b: Status 404 returned error can't find the container with id 751bf5d185e83ddaedb0d4433d917c9b9ba1822311f135611b8c90ffd8bca42b
Sep 17 14:09:15.915055 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:15.915024    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:09:16.302331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:16.302241    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz" event={"ID":"830200e7-15e8-4dba-99be-0dd1cacf8f29","Type":"ContainerStarted","Data":"751bf5d185e83ddaedb0d4433d917c9b9ba1822311f135611b8c90ffd8bca42b"}
Sep 17 14:09:17.308886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:17.308860    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz" event={"ID":"830200e7-15e8-4dba-99be-0dd1cacf8f29","Type":"ContainerStarted","Data":"4476d2a3eb5fc8ee9c9919691bdd1a790739608e74d4b7d46954dddaf23c5638"}
Sep 17 14:09:17.331417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:17.331373    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz" podStartSLOduration=2.2653013570000002 podStartE2EDuration="3.331356995s" totalImagesPullingTime="1.066055638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:09:17.328878509 +0000 UTC m=+799.565755989" watchObservedRunningTime="2026-09-17 14:09:17.331356995 +0000 UTC m=+799.568234469"
Sep 17 14:09:25.224124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:25.224090    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:25.224608 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:25.224172    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:25.236570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:25.236548    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:25.355981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:25.355941    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz"
Sep 17 14:09:30.714833 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:30.714796    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:09:38.224676 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:38.224639    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:09:53.819180 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:09:53.819145    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:10:13.602209 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.602176    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"]
Sep 17 14:10:13.605701 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.605683    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.608630 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.608605    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 17 14:10:13.615015 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.614994    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"]
Sep 17 14:10:13.764194 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.764348 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764206    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.764348 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.764348 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e634a448-c17e-401f-bbc2-662720ec54da-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.764499 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.764499 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.764392    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f56xx\" (UniqueName: \"kubernetes.io/projected/e634a448-c17e-401f-bbc2-662720ec54da-kube-api-access-f56xx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865474 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865474 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865474 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865719 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865498    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e634a448-c17e-401f-bbc2-662720ec54da-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865719 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865719 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865592    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f56xx\" (UniqueName: \"kubernetes.io/projected/e634a448-c17e-401f-bbc2-662720ec54da-kube-api-access-f56xx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865913 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865892    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.865974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.865929    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.866057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.866030    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.867908 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.867888    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e634a448-c17e-401f-bbc2-662720ec54da-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.868170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.868153    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e634a448-c17e-401f-bbc2-662720ec54da-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.874113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.874088    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f56xx\" (UniqueName: \"kubernetes.io/projected/e634a448-c17e-401f-bbc2-662720ec54da-kube-api-access-f56xx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-btx67\" (UID: \"e634a448-c17e-401f-bbc2-662720ec54da\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:13.916142 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:13.916113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:14.189272 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:14.189253    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"]
Sep 17 14:10:14.191208 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:10:14.191180    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode634a448_c17e_401f_bbc2_662720ec54da.slice/crio-63a0d63c6a47e82b9f9dce0264c7d6ca1190833747cc027b63dd578eb71f0c7b WatchSource:0}: Error finding container 63a0d63c6a47e82b9f9dce0264c7d6ca1190833747cc027b63dd578eb71f0c7b: Status 404 returned error can't find the container with id 63a0d63c6a47e82b9f9dce0264c7d6ca1190833747cc027b63dd578eb71f0c7b
Sep 17 14:10:14.192886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:14.192853    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:10:14.568786 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:14.568721    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67" event={"ID":"e634a448-c17e-401f-bbc2-662720ec54da","Type":"ContainerStarted","Data":"871a65152c3a2c551bd334fd6749e63d66cdd68c8163a01be00298f3ca3607e1"}
Sep 17 14:10:14.568955 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:14.568794    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67" event={"ID":"e634a448-c17e-401f-bbc2-662720ec54da","Type":"ContainerStarted","Data":"63a0d63c6a47e82b9f9dce0264c7d6ca1190833747cc027b63dd578eb71f0c7b"}
Sep 17 14:10:14.585503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:14.585457    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67" podStartSLOduration=1.383020235 podStartE2EDuration="1.585443204s" totalImagesPullingTime="202.422969ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:10:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:10:14.584453317 +0000 UTC m=+856.821330794" watchObservedRunningTime="2026-09-17 14:10:14.585443204 +0000 UTC m=+856.822320682"
Sep 17 14:10:15.960029 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:15.959994    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:10:23.916456 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:23.916411    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:23.917012 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:23.916468    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:23.929038 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:23.929011    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:10:24.634155 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:10:24.634128    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-btx67"
Sep 17 14:11:08.251013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.246773    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cf6768ff4-fj9t6"]
Sep 17 14:11:08.252609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.252585    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.258598 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.258574    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cf6768ff4-fj9t6"]
Sep 17 14:11:08.308450 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.308421    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/889e95ac-aaee-4a69-af31-faae509e2548-tls-cert\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.308608 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.308484    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8v6\" (UniqueName: \"kubernetes.io/projected/889e95ac-aaee-4a69-af31-faae509e2548-kube-api-access-xw8v6\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.409066 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.409033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8v6\" (UniqueName: \"kubernetes.io/projected/889e95ac-aaee-4a69-af31-faae509e2548-kube-api-access-xw8v6\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.409214 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.409094    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/889e95ac-aaee-4a69-af31-faae509e2548-tls-cert\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.411624 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.411598    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/889e95ac-aaee-4a69-af31-faae509e2548-tls-cert\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.417717 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.417693    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8v6\" (UniqueName: \"kubernetes.io/projected/889e95ac-aaee-4a69-af31-faae509e2548-kube-api-access-xw8v6\") pod \"authorino-5cf6768ff4-fj9t6\" (UID: \"889e95ac-aaee-4a69-af31-faae509e2548\") " pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.563695 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.563610    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cf6768ff4-fj9t6"
Sep 17 14:11:08.835617 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:08.835596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cf6768ff4-fj9t6"]
Sep 17 14:11:08.837703 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:08.837677    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod889e95ac_aaee_4a69_af31_faae509e2548.slice/crio-64dafc956b77a64df279f8459431303b61605bb18ce178d960a1a520c71e606e WatchSource:0}: Error finding container 64dafc956b77a64df279f8459431303b61605bb18ce178d960a1a520c71e606e: Status 404 returned error can't find the container with id 64dafc956b77a64df279f8459431303b61605bb18ce178d960a1a520c71e606e
Sep 17 14:11:09.836828 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:09.836713    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf6768ff4-fj9t6" event={"ID":"889e95ac-aaee-4a69-af31-faae509e2548","Type":"ContainerStarted","Data":"1cc595ca0fc15f018a21d9e205152b11d12c9f25aa8fe9e8a856ecd3c79a51a6"}
Sep 17 14:11:09.836828 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:09.836786    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cf6768ff4-fj9t6" event={"ID":"889e95ac-aaee-4a69-af31-faae509e2548","Type":"ContainerStarted","Data":"64dafc956b77a64df279f8459431303b61605bb18ce178d960a1a520c71e606e"}
Sep 17 14:11:09.861349 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:09.861290    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cf6768ff4-fj9t6" podStartSLOduration=1.286862821 podStartE2EDuration="1.86124643s" totalImagesPullingTime="574.383609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:09.858568533 +0000 UTC m=+912.095446012" watchObservedRunningTime="2026-09-17 14:11:09.86124643 +0000 UTC m=+912.098123907"
Sep 17 14:11:09.885115 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:09.885082    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:11:09.885327 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:09.885302    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" podUID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" containerName="authorino" containerID="cri-o://32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f" gracePeriod=30
Sep 17 14:11:10.142543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.142512    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:11:10.237543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.237502    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert\") pod \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") "
Sep 17 14:11:10.237774 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.237555    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbvgx\" (UniqueName: \"kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx\") pod \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\" (UID: \"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc\") "
Sep 17 14:11:10.239844 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.239809    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx" (OuterVolumeSpecName: "kube-api-access-cbvgx") pod "45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" (UID: "45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc"). InnerVolumeSpecName "kube-api-access-cbvgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:11:10.249015 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.248977    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" (UID: "45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:11:10.338561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.338532    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbvgx\" (UniqueName: \"kubernetes.io/projected/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-kube-api-access-cbvgx\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:11:10.338561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.338560    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc-tls-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:11:10.844302 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.844218    2570 generic.go:356] "Generic (PLEG): container finished" podID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" containerID="32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f" exitCode=0
Sep 17 14:11:10.844302 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.844270    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd"
Sep 17 14:11:10.844770 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.844314    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" event={"ID":"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc","Type":"ContainerDied","Data":"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"}
Sep 17 14:11:10.844770 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.844350    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ccdbd7b7-tj4nd" event={"ID":"45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc","Type":"ContainerDied","Data":"7593190a3fd9bc013faee2bd7d6d5c0eb82fca4cbff2b97934727796763b59b6"}
Sep 17 14:11:10.844770 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.844369    2570 scope.go:117] "RemoveContainer" containerID="32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"
Sep 17 14:11:10.853546 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.853528    2570 scope.go:117] "RemoveContainer" containerID="32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"
Sep 17 14:11:10.853856 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:10.853838    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f\": container with ID starting with 32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f not found: ID does not exist" containerID="32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"
Sep 17 14:11:10.853918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.853865    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f"} err="failed to get container status \"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f\": rpc error: code = NotFound desc = could not find container \"32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f\": container with ID starting with 32bc95e9a5dad24763f3f76be3239cafa60ff329340c2ea4e5b0065756694a3f not found: ID does not exist"
Sep 17 14:11:10.867575 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.867552    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:11:10.871685 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:10.871662    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5ccdbd7b7-tj4nd"]
Sep 17 14:11:12.181501 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:12.181468    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" path="/var/lib/kubelet/pods/45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc/volumes"
Sep 17 14:11:27.624935 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.624883    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:11:27.625328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.625243    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" containerName="authorino"
Sep 17 14:11:27.625328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.625254    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" containerName="authorino"
Sep 17 14:11:27.625328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.625319    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b7b6fb-dd8d-44c8-abd4-c8a347bc6bcc" containerName="authorino"
Sep 17 14:11:27.628235 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.628218    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.630980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.630958    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-079b38-openshift-default-dockercfg-c62fg\""
Sep 17 14:11:27.641860 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.641841    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:11:27.690654 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690621    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.690853 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690681    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.690853 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.690853 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690799    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.691028 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.691028 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.690960    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnltn\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.691028 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.691019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.691170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.691052    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.691170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.691077    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791412 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791377    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791484    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791870 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791632    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791870 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791800    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.791870 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791839    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.792023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791875    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.792106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.792082    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.792166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.791665    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnltn\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.792219 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.792206    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.792271 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.792226    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.794519 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.794486    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.794648 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.794572    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.805787 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.805756    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.806005 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.805984    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnltn\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn\") pod \"e2e-worker-w4-079b38-openshift-default-778d985f94-q6949\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") " pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:27.940074 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:27.940047    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:28.226545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.226520    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:11:28.228345 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:28.228317    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982631a4_7721_44f1_9032_377754e77c64.slice/crio-ee7954015621a7e389f0d081f74621f15efbcb8d8968ca878297b3912e7171ce WatchSource:0}: Error finding container ee7954015621a7e389f0d081f74621f15efbcb8d8968ca878297b3912e7171ce: Status 404 returned error can't find the container with id ee7954015621a7e389f0d081f74621f15efbcb8d8968ca878297b3912e7171ce
Sep 17 14:11:28.230500 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.230468    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:28.230591 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.230533    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:28.230591 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.230559    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:28.931141 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.931105    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerStarted","Data":"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"}
Sep 17 14:11:28.931573 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.931160    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerStarted","Data":"ee7954015621a7e389f0d081f74621f15efbcb8d8968ca878297b3912e7171ce"}
Sep 17 14:11:28.940193 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.940171    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:28.971323 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:28.971268    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podStartSLOduration=1.971249432 podStartE2EDuration="1.971249432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:28.967271611 +0000 UTC m=+931.204149087" watchObservedRunningTime="2026-09-17 14:11:28.971249432 +0000 UTC m=+931.208126909"
Sep 17 14:11:29.134699 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.134666    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7786-a13abc"]
Sep 17 14:11:29.137936 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.137919    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:29.140847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.140814    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f8z2f\""
Sep 17 14:11:29.143363 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.143339    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7786-a13abc"]
Sep 17 14:11:29.205068 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.204983    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l852\" (UniqueName: \"kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852\") pod \"test-curl-7786-a13abc\" (UID: \"7b197fe2-7817-4cd2-98c1-cd8008241220\") " pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:29.306158 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.306118    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l852\" (UniqueName: \"kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852\") pod \"test-curl-7786-a13abc\" (UID: \"7b197fe2-7817-4cd2-98c1-cd8008241220\") " pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:29.315624 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.315590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l852\" (UniqueName: \"kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852\") pod \"test-curl-7786-a13abc\" (UID: \"7b197fe2-7817-4cd2-98c1-cd8008241220\") " pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:29.448861 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.448810    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:29.720213 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.720186    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7786-a13abc"]
Sep 17 14:11:29.722215 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:29.722186    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b197fe2_7817_4cd2_98c1_cd8008241220.slice/crio-66e45ddc9c11b5826a203cd8b9ff10caf43c264858f497f813c1f4045046bbba WatchSource:0}: Error finding container 66e45ddc9c11b5826a203cd8b9ff10caf43c264858f497f813c1f4045046bbba: Status 404 returned error can't find the container with id 66e45ddc9c11b5826a203cd8b9ff10caf43c264858f497f813c1f4045046bbba
Sep 17 14:11:29.937367 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.937324    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7786-a13abc" event={"ID":"7b197fe2-7817-4cd2-98c1-cd8008241220","Type":"ContainerStarted","Data":"66e45ddc9c11b5826a203cd8b9ff10caf43c264858f497f813c1f4045046bbba"}
Sep 17 14:11:29.946576 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:29.946556    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:30.942872 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:30.942841    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:30.944011 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:30.943987    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:31.459367 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:31.459330    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:11:32.956424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:32.956378    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7786-a13abc" event={"ID":"7b197fe2-7817-4cd2-98c1-cd8008241220","Type":"ContainerStarted","Data":"bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be"}
Sep 17 14:11:32.986656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:32.986597    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7786-a13abc" podStartSLOduration=1.377249282 podStartE2EDuration="3.986579533s" totalImagesPullingTime="2.609330251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:32.984813125 +0000 UTC m=+935.221690604" watchObservedRunningTime="2026-09-17 14:11:32.986579533 +0000 UTC m=+935.223457010"
Sep 17 14:11:33.582014 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.581965    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7786-a13abc"]
Sep 17 14:11:33.584439 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.584403    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7786-a13abc"]
Sep 17 14:11:33.699512 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.699483    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7786-415f8f"]
Sep 17 14:11:33.699878 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.699864    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" containerName="test-curl-7786-a13abc"
Sep 17 14:11:33.699927 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.699880    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" containerName="test-curl-7786-a13abc"
Sep 17 14:11:33.699961 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.699933    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" containerName="test-curl-7786-a13abc"
Sep 17 14:11:33.703130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.703114    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:33.710948 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.710919    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7786-415f8f"]
Sep 17 14:11:33.748965 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.748926    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrpqx\" (UniqueName: \"kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx\") pod \"test-curl-7786-415f8f\" (UID: \"04d81342-9b2f-44db-8770-61f87d7f2120\") " pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:33.849531 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.849496    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrpqx\" (UniqueName: \"kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx\") pod \"test-curl-7786-415f8f\" (UID: \"04d81342-9b2f-44db-8770-61f87d7f2120\") " pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:33.859672 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:33.859637    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrpqx\" (UniqueName: \"kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx\") pod \"test-curl-7786-415f8f\" (UID: \"04d81342-9b2f-44db-8770-61f87d7f2120\") " pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:33.961286 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:33.961208    2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7786-a13abc" container="test-curl-7786-a13abc"
Sep 17 14:11:34.013328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.013299    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:34.285044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.285022    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7786-415f8f"]
Sep 17 14:11:34.287131 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:34.287101    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04d81342_9b2f_44db_8770_61f87d7f2120.slice/crio-127c1c8c1652e386a276181735dab9052bce67beb68cdbccf4af3e4c43fab4cf WatchSource:0}: Error finding container 127c1c8c1652e386a276181735dab9052bce67beb68cdbccf4af3e4c43fab4cf: Status 404 returned error can't find the container with id 127c1c8c1652e386a276181735dab9052bce67beb68cdbccf4af3e4c43fab4cf
Sep 17 14:11:34.892338 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.892307    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:11:34.896252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.896229    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:34.900259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.900235    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 14:11:34.900370 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.900264    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-j4j8k\""
Sep 17 14:11:34.900370 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.900272    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-96cfc6-serving-cert\""
Sep 17 14:11:34.900776 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.900263    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 14:11:34.900885 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.900793    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 14:11:34.912867 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.908329    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:11:34.958974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.958943    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrhb\" (UniqueName: \"kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:34.959119 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.958997    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:34.959119 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.959075    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:34.959214 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.959148    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:34.967222 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.967189    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7786-415f8f" event={"ID":"04d81342-9b2f-44db-8770-61f87d7f2120","Type":"ContainerStarted","Data":"ae9532299d470738d06a8724322873b20066374d1757298d8c7bdc773eaf097e"}
Sep 17 14:11:34.967609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.967233    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7786-415f8f" event={"ID":"04d81342-9b2f-44db-8770-61f87d7f2120","Type":"ContainerStarted","Data":"127c1c8c1652e386a276181735dab9052bce67beb68cdbccf4af3e4c43fab4cf"}
Sep 17 14:11:34.967609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.967279    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7786-a13abc" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" containerName="test-curl-7786-a13abc" containerID="cri-o://bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be" gracePeriod=2
Sep 17 14:11:34.989363 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:34.989312    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7786-415f8f" podStartSLOduration=1.98929432 podStartE2EDuration="1.98929432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:34.987471135 +0000 UTC m=+937.224348612" watchObservedRunningTime="2026-09-17 14:11:34.98929432 +0000 UTC m=+937.226171798"
Sep 17 14:11:35.059904 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.059868    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.060060 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.059912    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.060060 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.059972    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrhb\" (UniqueName: \"kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.060060 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.060031    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.060213 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:35.060094    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-serving-cert: secret "maas-api-e2e-inf-a-96cfc6-serving-cert" not found
Sep 17 14:11:35.060213 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:35.060183    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls podName:6574ab04-15cc-40aa-a3ec-788e6d692992 nodeName:}" failed. No retries permitted until 2026-09-17 14:11:35.560162177 +0000 UTC m=+937.797039635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls") pod "maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" (UID: "6574ab04-15cc-40aa-a3ec-788e6d692992") : secret "maas-api-e2e-inf-a-96cfc6-serving-cert" not found
Sep 17 14:11:35.060617 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.060599    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.062564 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.062542    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.069191 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.069145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrhb\" (UniqueName: \"kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.564542 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.564506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.566961 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.566938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") pod \"maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.597397 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.597353    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7786-415f8f"]
Sep 17 14:11:35.600312 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.600288    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7786-415f8f"]
Sep 17 14:11:35.681356 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.681326    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:11:35.681679 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.681666    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d81342-9b2f-44db-8770-61f87d7f2120" containerName="test-curl-7786-415f8f"
Sep 17 14:11:35.681762 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.681680    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d81342-9b2f-44db-8770-61f87d7f2120" containerName="test-curl-7786-415f8f"
Sep 17 14:11:35.681818 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.681765    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d81342-9b2f-44db-8770-61f87d7f2120" containerName="test-curl-7786-415f8f"
Sep 17 14:11:35.685080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.685062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.687839 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.687821    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-96cfc6\""
Sep 17 14:11:35.687918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.687822    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-96cfc6-dockercfg-k98h9\""
Sep 17 14:11:35.695324 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.695303    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:11:35.767028 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.767002    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28z6c\" (UniqueName: \"kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.767170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.767040    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.814547 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.814522    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:35.868085 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.868049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.868231 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.868204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28z6c\" (UniqueName: \"kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.870320 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.869356    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.883596 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.883571    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28z6c\" (UniqueName: \"kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c\") pod \"payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:35.972527 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:35.972439    2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7786-415f8f" container="test-curl-7786-415f8f"
Sep 17 14:11:35.995984 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:35.995958    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:36.092939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.092882    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:11:36.095759 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:36.095707    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6574ab04_15cc_40aa_a3ec_788e6d692992.slice/crio-ec91a593bee71c6676008adf93cfbce85e33234c97d95c67176384e770557ab6 WatchSource:0}: Error finding container ec91a593bee71c6676008adf93cfbce85e33234c97d95c67176384e770557ab6: Status 404 returned error can't find the container with id ec91a593bee71c6676008adf93cfbce85e33234c97d95c67176384e770557ab6
Sep 17 14:11:36.290138 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:36.290109    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591b56ff_da2c_4201_8142_7bb0c4f33201.slice/crio-23123ef70ca80137dc70277d35a43b5269cf0851eaed6b1af4d9a2ebb73e5f97 WatchSource:0}: Error finding container 23123ef70ca80137dc70277d35a43b5269cf0851eaed6b1af4d9a2ebb73e5f97: Status 404 returned error can't find the container with id 23123ef70ca80137dc70277d35a43b5269cf0851eaed6b1af4d9a2ebb73e5f97
Sep 17 14:11:36.298528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.298502    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:11:36.484245 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.484213    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:11:36.490038 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.490015    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.493035 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.493008    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-0458f0-openshift-default-dockercfg-ptwjs\""
Sep 17 14:11:36.501132 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.501111    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:11:36.576154 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576081    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk5mq\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576154 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576136    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576161    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576197    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576259    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576324    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576345    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.576447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.576365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677686 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677686 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677640    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677702    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677776    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677815    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677840    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677878    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677902    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.677980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.677959    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk5mq\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.678622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.678059    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.678622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.678352    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.678622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.678531    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.678622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.678546    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.678622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.678590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.681204 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.681169    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.682938 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.682897    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.687058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.687036    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.687424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.687392    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk5mq\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq\") pod \"e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") " pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.802069 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.801946    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:36.983229 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.983067    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" event={"ID":"591b56ff-da2c-4201-8142-7bb0c4f33201","Type":"ContainerStarted","Data":"23123ef70ca80137dc70277d35a43b5269cf0851eaed6b1af4d9a2ebb73e5f97"}
Sep 17 14:11:36.985683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.985579    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" event={"ID":"6574ab04-15cc-40aa-a3ec-788e6d692992","Type":"ContainerStarted","Data":"ec91a593bee71c6676008adf93cfbce85e33234c97d95c67176384e770557ab6"}
Sep 17 14:11:36.985683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:36.985648    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7786-415f8f" podUID="04d81342-9b2f-44db-8770-61f87d7f2120" containerName="test-curl-7786-415f8f" containerID="cri-o://ae9532299d470738d06a8724322873b20066374d1757298d8c7bdc773eaf097e" gracePeriod=2
Sep 17 14:11:37.123166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.123104    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:11:37.138859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.138165    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:37.138859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.138248    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:37.138859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.138287    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:37.212411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.212386    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:37.285178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.283998    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l852\" (UniqueName: \"kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852\") pod \"7b197fe2-7817-4cd2-98c1-cd8008241220\" (UID: \"7b197fe2-7817-4cd2-98c1-cd8008241220\") "
Sep 17 14:11:37.287650 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.287603    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852" (OuterVolumeSpecName: "kube-api-access-9l852") pod "7b197fe2-7817-4cd2-98c1-cd8008241220" (UID: "7b197fe2-7817-4cd2-98c1-cd8008241220"). InnerVolumeSpecName "kube-api-access-9l852". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:11:37.385136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.385050    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l852\" (UniqueName: \"kubernetes.io/projected/7b197fe2-7817-4cd2-98c1-cd8008241220-kube-api-access-9l852\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:11:37.993120 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.993089    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerStarted","Data":"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"}
Sep 17 14:11:37.993568 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:37.993140    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerStarted","Data":"8bfdd67bc0cc04dcff0fc7349f9458e6661bb7059cc48187241d35ee9be6cf4c"}
Sep 17 14:11:38.001855 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.001815    2570 generic.go:356] "Generic (PLEG): container finished" podID="7b197fe2-7817-4cd2-98c1-cd8008241220" containerID="bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be" exitCode=137
Sep 17 14:11:38.002003 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.001882    2570 scope.go:117] "RemoveContainer" containerID="bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be"
Sep 17 14:11:38.002100 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.002084    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:38.017805 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.017760    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podStartSLOduration=2.017744124 podStartE2EDuration="2.017744124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:38.016570464 +0000 UTC m=+940.253447942" watchObservedRunningTime="2026-09-17 14:11:38.017744124 +0000 UTC m=+940.254621596"
Sep 17 14:11:38.018803 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:38.018772    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7786-a13abc\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:38.183857 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:38.183800    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7786-a13abc\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" pod="opendatahub/test-curl-7786-a13abc"
Sep 17 14:11:38.187122 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.187089    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b197fe2-7817-4cd2-98c1-cd8008241220" path="/var/lib/kubelet/pods/7b197fe2-7817-4cd2-98c1-cd8008241220/volumes"
Sep 17 14:11:38.802702 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.802663    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:38.809683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.809656    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:38.946212 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.946049    2570 scope.go:117] "RemoveContainer" containerID="bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be"
Sep 17 14:11:38.946432 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:38.946411    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be\": container with ID starting with bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be not found: ID does not exist" containerID="bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be"
Sep 17 14:11:38.946482 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:38.946441    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be"} err="failed to get container status \"bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be\": rpc error: code = NotFound desc = could not find container \"bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be\": container with ID starting with bf6b848cb292920eb04f2a7d3792f52882921e30c8e6ee1b0ba329ca355f70be not found: ID does not exist"
Sep 17 14:11:39.009304 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:39.009273    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:39.010458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:39.010429    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:11:40.015720 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:40.015647    2570 generic.go:356] "Generic (PLEG): container finished" podID="04d81342-9b2f-44db-8770-61f87d7f2120" containerID="ae9532299d470738d06a8724322873b20066374d1757298d8c7bdc773eaf097e" exitCode=137
Sep 17 14:11:40.164665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:40.164607    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:40.213354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:40.213322    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrpqx\" (UniqueName: \"kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx\") pod \"04d81342-9b2f-44db-8770-61f87d7f2120\" (UID: \"04d81342-9b2f-44db-8770-61f87d7f2120\") "
Sep 17 14:11:40.215522 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:40.215489    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx" (OuterVolumeSpecName: "kube-api-access-zrpqx") pod "04d81342-9b2f-44db-8770-61f87d7f2120" (UID: "04d81342-9b2f-44db-8770-61f87d7f2120"). InnerVolumeSpecName "kube-api-access-zrpqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:11:40.314252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:40.314129    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrpqx\" (UniqueName: \"kubernetes.io/projected/04d81342-9b2f-44db-8770-61f87d7f2120-kube-api-access-zrpqx\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:11:41.024255 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.024175    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7786-415f8f"
Sep 17 14:11:41.024255 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.024201    2570 scope.go:117] "RemoveContainer" containerID="ae9532299d470738d06a8724322873b20066374d1757298d8c7bdc773eaf097e"
Sep 17 14:11:41.026468 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.026430    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" event={"ID":"591b56ff-da2c-4201-8142-7bb0c4f33201","Type":"ContainerStarted","Data":"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"}
Sep 17 14:11:41.026664 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.026639    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:41.028636 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.028612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" event={"ID":"6574ab04-15cc-40aa-a3ec-788e6d692992","Type":"ContainerStarted","Data":"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"}
Sep 17 14:11:41.047381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.047343    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" podStartSLOduration=2.258375039 podStartE2EDuration="6.047329015s" totalImagesPullingTime="3.788953976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:41.046627439 +0000 UTC m=+943.283504918" watchObservedRunningTime="2026-09-17 14:11:41.047329015 +0000 UTC m=+943.284206492"
Sep 17 14:11:41.070072 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:41.070023    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" podStartSLOduration=3.134570287 podStartE2EDuration="7.070003465s" totalImagesPullingTime="3.935433178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:41.067330041 +0000 UTC m=+943.304207520" watchObservedRunningTime="2026-09-17 14:11:41.070003465 +0000 UTC m=+943.306880944"
Sep 17 14:11:42.035899 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:42.035868    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:42.196788 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:42.196753    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d81342-9b2f-44db-8770-61f87d7f2120" path="/var/lib/kubelet/pods/04d81342-9b2f-44db-8770-61f87d7f2120/volumes"
Sep 17 14:11:43.513358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.513320    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:11:43.518790 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.518765    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.522098 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.522076    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1ee55ce9-serving-cert\""
Sep 17 14:11:43.529199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.529175    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:11:43.643589 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.643560    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.643724 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.643602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.643795 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.643755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76z66\" (UniqueName: \"kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.643837 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.643824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.745196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.745163    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.745196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.745201    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.745431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.745223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.745431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.745273    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76z66\" (UniqueName: \"kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.745431 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:43.745338    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-serving-cert: secret "maas-api-e2e-ait-1ee55ce9-serving-cert" not found
Sep 17 14:11:43.745431 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:11:43.745431    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls podName:8a751298-abc9-427f-a233-dbc6ad8f5db3 nodeName:}" failed. No retries permitted until 2026-09-17 14:11:44.245409426 +0000 UTC m=+946.482286899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls") pod "maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" (UID: "8a751298-abc9-427f-a233-dbc6ad8f5db3") : secret "maas-api-e2e-ait-1ee55ce9-serving-cert" not found
Sep 17 14:11:43.746063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.746040    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.747779 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.747762    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:43.758859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:43.758838    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76z66\" (UniqueName: \"kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:44.248821 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:44.248786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:44.257253 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:44.257225    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") pod \"maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:44.431545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:44.431499    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:44.736992 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:44.736970    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:11:44.739178 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:44.739156    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a751298_abc9_427f_a233_dbc6ad8f5db3.slice/crio-1006b6316c555983ca168eff4f937d7af9dbf3be26d9da41fbca255a5490cc38 WatchSource:0}: Error finding container 1006b6316c555983ca168eff4f937d7af9dbf3be26d9da41fbca255a5490cc38: Status 404 returned error can't find the container with id 1006b6316c555983ca168eff4f937d7af9dbf3be26d9da41fbca255a5490cc38
Sep 17 14:11:45.053401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.053304    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" event={"ID":"8a751298-abc9-427f-a233-dbc6ad8f5db3","Type":"ContainerStarted","Data":"1006b6316c555983ca168eff4f937d7af9dbf3be26d9da41fbca255a5490cc38"}
Sep 17 14:11:45.177011 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.176964    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:11:45.181623 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.181520    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.184186 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.184164    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0458f0\""
Sep 17 14:11:45.184308 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.184199    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0458f0-dockercfg-l6hsm\""
Sep 17 14:11:45.195857 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.195833    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:11:45.276754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.273876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5qnl\" (UniqueName: \"kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.276754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.273932    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.375401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.375362    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5qnl\" (UniqueName: \"kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.375574 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.375415    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.377371 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.377340    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.385021 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.384999    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5qnl\" (UniqueName: \"kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl\") pod \"payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.492875 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.492843    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:45.802104 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:45.802071    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:11:45.804430 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:45.804404    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54bdcf0e_2bcc_4ea0_bda9_cacf24547ab8.slice/crio-83e4e060cca23a55f3c2ae89b8a172611ff747ce0227b1a9a28844a2723ee8b6 WatchSource:0}: Error finding container 83e4e060cca23a55f3c2ae89b8a172611ff747ce0227b1a9a28844a2723ee8b6: Status 404 returned error can't find the container with id 83e4e060cca23a55f3c2ae89b8a172611ff747ce0227b1a9a28844a2723ee8b6
Sep 17 14:11:46.062860 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.062804    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" event={"ID":"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8","Type":"ContainerStarted","Data":"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"}
Sep 17 14:11:46.063054 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.062907    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" event={"ID":"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8","Type":"ContainerStarted","Data":"83e4e060cca23a55f3c2ae89b8a172611ff747ce0227b1a9a28844a2723ee8b6"}
Sep 17 14:11:46.063901 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.063875    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:46.066034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.066006    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" event={"ID":"8a751298-abc9-427f-a233-dbc6ad8f5db3","Type":"ContainerStarted","Data":"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"}
Sep 17 14:11:46.066650 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.066629    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:46.084104 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.083836    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" podStartSLOduration=1.08382373 podStartE2EDuration="1.08382373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:46.081840199 +0000 UTC m=+948.318717675" watchObservedRunningTime="2026-09-17 14:11:46.08382373 +0000 UTC m=+948.320701207"
Sep 17 14:11:46.102292 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:46.102218    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" podStartSLOduration=2.6344464370000003 podStartE2EDuration="3.102205581s" totalImagesPullingTime="467.759144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:46.099764512 +0000 UTC m=+948.336641989" watchObservedRunningTime="2026-09-17 14:11:46.102205581 +0000 UTC m=+948.339083081"
Sep 17 14:11:48.047413 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:48.047381    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:11:51.099633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.099603    2570 generic.go:356] "Generic (PLEG): container finished" podID="982631a4-7721-44f1-9032-377754e77c64" containerID="6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037" exitCode=0
Sep 17 14:11:51.100104 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.099674    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerDied","Data":"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"}
Sep 17 14:11:51.100154 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.100134    2570 scope.go:117] "RemoveContainer" containerID="6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"
Sep 17 14:11:51.101419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.101387    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:51.101516 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.101458    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:51.101516 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.101497    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:51.814786 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.814753    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:11:51.817285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.817252    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.820317 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.820300    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-fe6ee5-openshift-default-dockercfg-b9hzs\""
Sep 17 14:11:51.831223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.831195    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:11:51.948647 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948608    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948658    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948710    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948799    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948815    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948834    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948849    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nk9q\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:51.948981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:51.948870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.037781 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.037749    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:11:52.049565 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.049696 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.049696 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049592    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.049847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049721    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.049847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.049847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049826    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nk9q\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.049966    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050156 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.050018    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050156 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.050043    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050392 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.050370    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050480 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.050376    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.050711 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.050687    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.056157 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.053183    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.056157 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.053468    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.063648 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.063622    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.063852 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.063834    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nk9q\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q\") pod \"e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") " pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.107357 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.107290    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerStarted","Data":"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"}
Sep 17 14:11:52.128675 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.128654    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:52.414672 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.414646    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:11:52.417592 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:52.417571    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8adbfc9f_822d_484a_8f53_60ae83e0e394.slice/crio-362de29b96fac96ff9c3208df0f57294a9f1f3257877bc335eb5b5af28f11f9c WatchSource:0}: Error finding container 362de29b96fac96ff9c3208df0f57294a9f1f3257877bc335eb5b5af28f11f9c: Status 404 returned error can't find the container with id 362de29b96fac96ff9c3208df0f57294a9f1f3257877bc335eb5b5af28f11f9c
Sep 17 14:11:52.420504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.420479    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:52.420644 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.420630    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:52.420725 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.420714    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:11:52.929599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.929569    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:11:52.941084 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.941061    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:52.941326 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.941299    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:52.941380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:52.941351    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:53.087559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.087532    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:11:53.115213 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.115181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerStarted","Data":"6230bc1b451f4017b2e00ce01c33e4cd6c104e2ea5bf6eaa38c192b53b8b1098"}
Sep 17 14:11:53.115572 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.115238    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerStarted","Data":"362de29b96fac96ff9c3208df0f57294a9f1f3257877bc335eb5b5af28f11f9c"}
Sep 17 14:11:53.129783 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.129596    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:53.135790 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.135764    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:53.143419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.143359    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podStartSLOduration=2.143338849 podStartE2EDuration="2.143338849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:53.140959012 +0000 UTC m=+955.377836549" watchObservedRunningTime="2026-09-17 14:11:53.143338849 +0000 UTC m=+955.380216339"
Sep 17 14:11:53.940590 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.940554    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:53.940776 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:53.940621    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:54.121249 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:54.121217    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:54.122248 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:54.122230    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:11:54.940941 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:54.940901    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:54.941850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:54.941796    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:55.940817 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:55.940786    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:55.941256 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:55.941234    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:56.940434 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:56.940395    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:56.940626 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:56.940452    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:57.940901 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:57.940860    2570 patch_prober.go:28] interesting pod/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 17 14:11:57.941258 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:57.940925    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 17 14:11:58.018632 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.018594    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:11:58.022893 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.022866    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.024886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.024860    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.025034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.024913    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lg4v\" (UniqueName: \"kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.025964 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.025938    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-fe6ee5-dockercfg-5bm9r\""
Sep 17 14:11:58.026479 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.026462    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-fe6ee5\""
Sep 17 14:11:58.035372 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.035334    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:11:58.087136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.085885    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:11:58.126436 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.126274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.126436 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.126338    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lg4v\" (UniqueName: \"kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.132314 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.132287    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-fe6ee5\""
Sep 17 14:11:58.137671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.137610    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.146759 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.146612    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lg4v\" (UniqueName: \"kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v\") pod \"payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.337488 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.337458    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-fe6ee5-dockercfg-5bm9r\""
Sep 17 14:11:58.345811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.345792    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:58.644107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.644078    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:11:58.646088 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:11:58.646065    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9198de_a972_4e6c_9d3c_0f41fb7a40ed.slice/crio-8810cebf76004ba1994508e8c47b13007226557f9ea015b88bd532ca6edbcd27 WatchSource:0}: Error finding container 8810cebf76004ba1994508e8c47b13007226557f9ea015b88bd532ca6edbcd27: Status 404 returned error can't find the container with id 8810cebf76004ba1994508e8c47b13007226557f9ea015b88bd532ca6edbcd27
Sep 17 14:11:58.944430 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.944404    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:58.944908 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.944725    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:58.945645 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:58.945627    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:11:59.148811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:59.148778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" event={"ID":"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed","Type":"ContainerStarted","Data":"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"}
Sep 17 14:11:59.148948 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:59.148836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" event={"ID":"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed","Type":"ContainerStarted","Data":"8810cebf76004ba1994508e8c47b13007226557f9ea015b88bd532ca6edbcd27"}
Sep 17 14:11:59.149058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:59.149043    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:11:59.185808 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:11:59.185725    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" podStartSLOduration=1.185704915 podStartE2EDuration="1.185704915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:11:59.185113118 +0000 UTC m=+961.421990595" watchObservedRunningTime="2026-09-17 14:11:59.185704915 +0000 UTC m=+961.422582396"
Sep 17 14:12:04.179205 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.179168    2570 generic.go:356] "Generic (PLEG): container finished" podID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerID="6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0" exitCode=0
Sep 17 14:12:04.181114 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.181092    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerDied","Data":"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"}
Sep 17 14:12:04.181479 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.181463    2570 scope.go:117] "RemoveContainer" containerID="6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"
Sep 17 14:12:04.182561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.182531    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:04.182634 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.182597    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:04.182634 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:04.182629    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:05.186344 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:05.186307    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerStarted","Data":"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"}
Sep 17 14:12:05.802814 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:05.802785    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:12:05.804034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:05.804008    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:05.804178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:05.804070    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:06.803048 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:06.803017    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:06.803475 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:06.803448    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:07.802969 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:07.802924    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:07.803229 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:07.803202    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:08.803331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:08.803296    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:08.803957 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:08.803921    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:09.009792 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:09.009756    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:12:09.802983 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:09.802948    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:09.803271 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:09.803240    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:10.159947 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.159918    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:12:10.598972 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.598881    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:12:10.602871 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.602849    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.607239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.607220    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d9b17d\"/\"simulator-w0-d9b17d-kserve-self-signed-certs\""
Sep 17 14:12:10.607356 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.607335    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d9b17d\"/\"openshift-service-ca.crt\""
Sep 17 14:12:10.607416 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.607303    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-d9b17d\"/\"kube-root-ca.crt\""
Sep 17 14:12:10.607511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.607280    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-d9b17d\"/\"default-dockercfg-xhl2t\""
Sep 17 14:12:10.611653 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.611637    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:12:10.634364 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634336    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzzt\" (UniqueName: \"kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.634468 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634390    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.634468 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634419    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.634563 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.634563 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634523    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.634644 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.634571    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.735854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.735819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.735854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.735854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.735899    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.735951    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzzt\" (UniqueName: \"kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.736003    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.736028    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736330 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.736307    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736330 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.736321    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.736408 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.736373    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.738351 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.738334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.738594 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.738577    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.745573 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.745542    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzzt\" (UniqueName: \"kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt\") pod \"simulator-w0-d9b17d-kserve-7445d774c4-frvdt\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") " pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:10.802643 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.802609    2570 patch_prober.go:28] interesting pod/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 17 14:12:10.802825 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.802663    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 17 14:12:10.914081 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:10.914045    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:11.215628 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.215606    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:12:11.218002 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:12:11.217974    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda07655d_388b_410d_b755_9b8d755d834e.slice/crio-6e6f6c8cb9a71779a2f0d47510b0f9547b2e6eb89c85cd74adaa59abca33510f WatchSource:0}: Error finding container 6e6f6c8cb9a71779a2f0d47510b0f9547b2e6eb89c85cd74adaa59abca33510f: Status 404 returned error can't find the container with id 6e6f6c8cb9a71779a2f0d47510b0f9547b2e6eb89c85cd74adaa59abca33510f
Sep 17 14:12:11.233797 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.233770    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" event={"ID":"da07655d-388b-410d-b755-9b8d755d834e","Type":"ContainerStarted","Data":"6e6f6c8cb9a71779a2f0d47510b0f9547b2e6eb89c85cd74adaa59abca33510f"}
Sep 17 14:12:11.807023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.806996    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:12:11.808040 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.808025    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:12:11.841584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.841129    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:12:11.841584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:11.841343    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" podUID="8a751298-abc9-427f-a233-dbc6ad8f5db3" containerName="maas-api" containerID="cri-o://b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23" gracePeriod=30
Sep 17 14:12:12.137456 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.137434    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:12:12.246131 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.246007    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" event={"ID":"da07655d-388b-410d-b755-9b8d755d834e","Type":"ContainerStarted","Data":"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"}
Sep 17 14:12:12.247813 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.247788    2570 generic.go:356] "Generic (PLEG): container finished" podID="8a751298-abc9-427f-a233-dbc6ad8f5db3" containerID="b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23" exitCode=0
Sep 17 14:12:12.248013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.247991    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" event={"ID":"8a751298-abc9-427f-a233-dbc6ad8f5db3","Type":"ContainerDied","Data":"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"}
Sep 17 14:12:12.248124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.248112    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"
Sep 17 14:12:12.248311 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.248127    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46" event={"ID":"8a751298-abc9-427f-a233-dbc6ad8f5db3","Type":"ContainerDied","Data":"1006b6316c555983ca168eff4f937d7af9dbf3be26d9da41fbca255a5490cc38"}
Sep 17 14:12:12.248395 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.248143    2570 scope.go:117] "RemoveContainer" containerID="b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"
Sep 17 14:12:12.256920 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.256889    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") pod \"8a751298-abc9-427f-a233-dbc6ad8f5db3\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") "
Sep 17 14:12:12.257037 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.256948    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca\") pod \"8a751298-abc9-427f-a233-dbc6ad8f5db3\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") "
Sep 17 14:12:12.257037 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.257003    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs\") pod \"8a751298-abc9-427f-a233-dbc6ad8f5db3\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") "
Sep 17 14:12:12.257158 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.257077    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76z66\" (UniqueName: \"kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66\") pod \"8a751298-abc9-427f-a233-dbc6ad8f5db3\" (UID: \"8a751298-abc9-427f-a233-dbc6ad8f5db3\") "
Sep 17 14:12:12.272254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.257980    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a751298-abc9-427f-a233-dbc6ad8f5db3" (UID: "8a751298-abc9-427f-a233-dbc6ad8f5db3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:12:12.275594 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.274488    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66" (OuterVolumeSpecName: "kube-api-access-76z66") pod "8a751298-abc9-427f-a233-dbc6ad8f5db3" (UID: "8a751298-abc9-427f-a233-dbc6ad8f5db3"). InnerVolumeSpecName "kube-api-access-76z66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:12:12.275944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.275922    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8a751298-abc9-427f-a233-dbc6ad8f5db3" (UID: "8a751298-abc9-427f-a233-dbc6ad8f5db3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:12:12.293894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.292079    2570 scope.go:117] "RemoveContainer" containerID="b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"
Sep 17 14:12:12.294293 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:12:12.294266    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23\": container with ID starting with b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23 not found: ID does not exist" containerID="b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"
Sep 17 14:12:12.294388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.294311    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23"} err="failed to get container status \"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23\": rpc error: code = NotFound desc = could not find container \"b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23\": container with ID starting with b0229732c5d25707e9bf78331cf09fadb8a9dcc67fa4fa7c4f6caaa8606fab23 not found: ID does not exist"
Sep 17 14:12:12.297304 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.297274    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a751298-abc9-427f-a233-dbc6ad8f5db3" (UID: "8a751298-abc9-427f-a233-dbc6ad8f5db3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:12:12.367799 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.365377    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:12.367799 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.365411    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a751298-abc9-427f-a233-dbc6ad8f5db3-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:12.367799 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.365426    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a751298-abc9-427f-a233-dbc6ad8f5db3-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:12.367799 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.365440    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76z66\" (UniqueName: \"kubernetes.io/projected/8a751298-abc9-427f-a233-dbc6ad8f5db3-kube-api-access-76z66\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:12.577897 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.577832    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" podStartSLOduration=2.577809184 podStartE2EDuration="2.577809184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:12.290891873 +0000 UTC m=+974.527769350" watchObservedRunningTime="2026-09-17 14:12:12.577809184 +0000 UTC m=+974.814686663"
Sep 17 14:12:12.578713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.578683    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:12:12.583213 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:12.583193    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1ee55ce9-85bdd9bd76-tdj46"]
Sep 17 14:12:14.183187 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:14.183149    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a751298-abc9-427f-a233-dbc6ad8f5db3" path="/var/lib/kubelet/pods/8a751298-abc9-427f-a233-dbc6ad8f5db3/volumes"
Sep 17 14:12:16.721023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:16.720984    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:12:17.280784 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.280748    2570 generic.go:356] "Generic (PLEG): container finished" podID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerID="6230bc1b451f4017b2e00ce01c33e4cd6c104e2ea5bf6eaa38c192b53b8b1098" exitCode=0
Sep 17 14:12:17.280784 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.280764    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerDied","Data":"6230bc1b451f4017b2e00ce01c33e4cd6c104e2ea5bf6eaa38c192b53b8b1098"}
Sep 17 14:12:17.281309 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.281291    2570 scope.go:117] "RemoveContainer" containerID="6230bc1b451f4017b2e00ce01c33e4cd6c104e2ea5bf6eaa38c192b53b8b1098"
Sep 17 14:12:17.282498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.282473    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:17.282599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.282576    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:17.282659 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:17.282624    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:12:18.288235 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.288197    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerStarted","Data":"46abae3a365952795ba45aa15f4104ae22df29dd5046d6b645ae8b46fc842986"}
Sep 17 14:12:18.431842 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.431816    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:18.432404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.432353    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a751298-abc9-427f-a233-dbc6ad8f5db3" containerName="maas-api"
Sep 17 14:12:18.432404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.432372    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a751298-abc9-427f-a233-dbc6ad8f5db3" containerName="maas-api"
Sep 17 14:12:18.432764 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.432443    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a751298-abc9-427f-a233-dbc6ad8f5db3" containerName="maas-api"
Sep 17 14:12:18.434470 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.434451    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.437657 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.437636    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-dd410a86-serving-cert\""
Sep 17 14:12:18.449300 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.449281    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:18.525054 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.525016    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njtfx\" (UniqueName: \"kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.525215 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.525081    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.525215 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.525157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.525215 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.525195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.625691 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.625604    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.625691 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.625642    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.625691 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.625666    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.625944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.625727    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njtfx\" (UniqueName: \"kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.625944 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:12:18.625851    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-serving-cert: secret "maas-api-e2e-migrate-dd410a86-serving-cert" not found
Sep 17 14:12:18.625944 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:12:18.625918    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls podName:3e227bf1-6855-4a34-9a2f-bf95ee3f4815 nodeName:}" failed. No retries permitted until 2026-09-17 14:12:19.125901233 +0000 UTC m=+981.362778692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls") pod "maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" (UID: "3e227bf1-6855-4a34-9a2f-bf95ee3f4815") : secret "maas-api-e2e-migrate-dd410a86-serving-cert" not found
Sep 17 14:12:18.626259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.626236    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.628124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.628098    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:18.636809 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:18.636776    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njtfx\" (UniqueName: \"kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:19.129179 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.129149    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:12:19.129376 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.129358    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 17 14:12:19.129453 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.129428    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 17 14:12:19.130887 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.130860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:19.133787 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.133767    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") pod \"maas-api-e2e-migrate-dd410a86-66998bb957-z84cc\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:19.296080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.296041    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:19.298488 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.298472    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.301902 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.301884    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-dd410a86\""
Sep 17 14:12:19.311551 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.311527    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:19.345228 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.345190    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:19.436565 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.435378    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7x5h\" (UniqueName: \"kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.436565 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.435440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.536520 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.536493    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.536764 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.536670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7x5h\" (UniqueName: \"kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.537201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.537181    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.560365 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.560311    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7x5h\" (UniqueName: \"kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h\") pod \"payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") " pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.609525 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.609499    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:19.664759 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.661172    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:19.919233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:19.919203    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:19.920805 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:12:19.920777    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c4ace8e_bb47_4aae_8584_804d82778410.slice/crio-028e8b4f447cae2e90e659eb089d7dabe8172048af3513557c482021f366c7b7 WatchSource:0}: Error finding container 028e8b4f447cae2e90e659eb089d7dabe8172048af3513557c482021f366c7b7: Status 404 returned error can't find the container with id 028e8b4f447cae2e90e659eb089d7dabe8172048af3513557c482021f366c7b7
Sep 17 14:12:20.129722 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.129699    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 17 14:12:20.129857 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.129760    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 17 14:12:20.300024 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.299932    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" event={"ID":"3c4ace8e-bb47-4aae-8584-804d82778410","Type":"ContainerStarted","Data":"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"}
Sep 17 14:12:20.300024 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.299988    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" event={"ID":"3c4ace8e-bb47-4aae-8584-804d82778410","Type":"ContainerStarted","Data":"028e8b4f447cae2e90e659eb089d7dabe8172048af3513557c482021f366c7b7"}
Sep 17 14:12:20.300504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.300110    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:20.301707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.301672    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" event={"ID":"3e227bf1-6855-4a34-9a2f-bf95ee3f4815","Type":"ContainerStarted","Data":"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"}
Sep 17 14:12:20.301836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.301754    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" event={"ID":"3e227bf1-6855-4a34-9a2f-bf95ee3f4815","Type":"ContainerStarted","Data":"d6e00eeca06764756e6aa0e470a3bbd165fb5193d46d02ea2646584caacfabd4"}
Sep 17 14:12:20.301836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.301785    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:20.318806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.318759    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" podStartSLOduration=1.318742489 podStartE2EDuration="1.318742489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:20.317822531 +0000 UTC m=+982.554700009" watchObservedRunningTime="2026-09-17 14:12:20.318742489 +0000 UTC m=+982.555619956"
Sep 17 14:12:20.336493 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.336454    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" podStartSLOduration=1.916432951 podStartE2EDuration="2.336440637s" totalImagesPullingTime="420.007686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:20.335432148 +0000 UTC m=+982.572309624" watchObservedRunningTime="2026-09-17 14:12:20.336440637 +0000 UTC m=+982.573318112"
Sep 17 14:12:20.914517 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.914485    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:20.914683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.914537    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:20.927993 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:20.927964    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:21.129027 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:21.128992    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 17 14:12:21.129194 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:21.129049    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 17 14:12:21.319118 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:21.319037    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:12:22.129236 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:22.129200    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 17 14:12:22.129406 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:22.129255    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 17 14:12:23.129918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:23.129869    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 17 14:12:23.130342 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:23.129937    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 17 14:12:24.121457 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:24.121425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:12:24.133121 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:24.133088    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:12:24.327191 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:24.327165    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:12:26.313809 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:26.313783    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:30.309570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.309539    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:12:30.312246 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.312224    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.316663 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.316645    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-079b38\"/\"default-dockercfg-xlhg2\""
Sep 17 14:12:30.316802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.316680    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-079b38\"/\"simulator-w4-079b38-kserve-self-signed-certs\""
Sep 17 14:12:30.316802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.316649    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-079b38\"/\"openshift-service-ca.crt\""
Sep 17 14:12:30.316927 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.316806    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-079b38\"/\"kube-root-ca.crt\""
Sep 17 14:12:30.325235 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.325215    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:12:30.457383 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.457348    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.457627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.457607    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.457755 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.457720    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.457954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.457936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.458097 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.458080    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97h96\" (UniqueName: \"kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.458203 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.458178    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559550 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559384    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559550 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559415    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559550 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559503    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559751 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97h96\" (UniqueName: \"kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559751 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559582    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559914 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559892    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.559980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.559938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.560041 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.560025    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.561828 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.561805    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.562133 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.562117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.572775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.572715    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97h96\" (UniqueName: \"kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96\") pod \"simulator-w4-079b38-kserve-764b944566-p9ztz\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") " pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.622600 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.622557    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:30.913673 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:30.913625    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:12:30.917186 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:12:30.917147    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec571499_5c08_4034_a391_164356db00c1.slice/crio-1950f4e3c1137425ce5eaeee3b19ffbceff49a0a7965906c23dab9eb318bc662 WatchSource:0}: Error finding container 1950f4e3c1137425ce5eaeee3b19ffbceff49a0a7965906c23dab9eb318bc662: Status 404 returned error can't find the container with id 1950f4e3c1137425ce5eaeee3b19ffbceff49a0a7965906c23dab9eb318bc662
Sep 17 14:12:31.310496 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:31.310403    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:31.368700 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:31.368662    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" event={"ID":"ec571499-5c08-4034-a391-164356db00c1","Type":"ContainerStarted","Data":"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"}
Sep 17 14:12:31.368700 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:31.368711    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" event={"ID":"ec571499-5c08-4034-a391-164356db00c1","Type":"ContainerStarted","Data":"1950f4e3c1137425ce5eaeee3b19ffbceff49a0a7965906c23dab9eb318bc662"}
Sep 17 14:12:31.391782 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:31.391716    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" podStartSLOduration=1.391699513 podStartE2EDuration="1.391699513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:31.390190089 +0000 UTC m=+993.627067567" watchObservedRunningTime="2026-09-17 14:12:31.391699513 +0000 UTC m=+993.628576989"
Sep 17 14:12:40.428173 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:40.428097    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:12:40.623232 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:40.623188    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:40.623431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:40.623255    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:40.636498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:40.636463    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:41.436478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:41.436451    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:12:47.084107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.084069    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:47.084460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.084325    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" podUID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" containerName="maas-api" containerID="cri-o://b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6" gracePeriod=30
Sep 17 14:12:47.149331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.149300    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:47.149522 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.149501    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" podUID="3c4ace8e-bb47-4aae-8584-804d82778410" containerName="payload-processing" containerID="cri-o://2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c" gracePeriod=30
Sep 17 14:12:47.413270 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.412326    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:47.462756 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.453091    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:47.467340 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.467292    2570 generic.go:356] "Generic (PLEG): container finished" podID="3c4ace8e-bb47-4aae-8584-804d82778410" containerID="2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c" exitCode=0
Sep 17 14:12:47.467512 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.467416    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"
Sep 17 14:12:47.467512 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.467431    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" event={"ID":"3c4ace8e-bb47-4aae-8584-804d82778410","Type":"ContainerDied","Data":"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"}
Sep 17 14:12:47.467625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.467517    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9" event={"ID":"3c4ace8e-bb47-4aae-8584-804d82778410","Type":"ContainerDied","Data":"028e8b4f447cae2e90e659eb089d7dabe8172048af3513557c482021f366c7b7"}
Sep 17 14:12:47.467625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.467543    2570 scope.go:117] "RemoveContainer" containerID="2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"
Sep 17 14:12:47.469467 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.469445    2570 generic.go:356] "Generic (PLEG): container finished" podID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" containerID="b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6" exitCode=0
Sep 17 14:12:47.469577 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.469498    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"
Sep 17 14:12:47.469640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.469578    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" event={"ID":"3e227bf1-6855-4a34-9a2f-bf95ee3f4815","Type":"ContainerDied","Data":"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"}
Sep 17 14:12:47.469640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.469623    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc" event={"ID":"3e227bf1-6855-4a34-9a2f-bf95ee3f4815","Type":"ContainerDied","Data":"d6e00eeca06764756e6aa0e470a3bbd165fb5193d46d02ea2646584caacfabd4"}
Sep 17 14:12:47.478628 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.478584    2570 scope.go:117] "RemoveContainer" containerID="2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"
Sep 17 14:12:47.478975 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:12:47.478949    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c\": container with ID starting with 2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c not found: ID does not exist" containerID="2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"
Sep 17 14:12:47.479113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.479089    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c"} err="failed to get container status \"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c\": rpc error: code = NotFound desc = could not find container \"2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c\": container with ID starting with 2e90e5b7c19651fee093e4bc7eeae12af3a87d952c42ff588c6b4738f80fc99c not found: ID does not exist"
Sep 17 14:12:47.479113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.479115    2570 scope.go:117] "RemoveContainer" containerID="b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"
Sep 17 14:12:47.491458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.491436    2570 scope.go:117] "RemoveContainer" containerID="b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"
Sep 17 14:12:47.491795 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:12:47.491775    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6\": container with ID starting with b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6 not found: ID does not exist" containerID="b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"
Sep 17 14:12:47.491910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.491802    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6"} err="failed to get container status \"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6\": rpc error: code = NotFound desc = could not find container \"b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6\": container with ID starting with b36197efe4780172b46fd2e2687fa80e51b3ab9fbaab0571564d6904913b92d6 not found: ID does not exist"
Sep 17 14:12:47.530295 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530268    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njtfx\" (UniqueName: \"kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx\") pod \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") "
Sep 17 14:12:47.530472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530304    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs\") pod \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") "
Sep 17 14:12:47.530472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530327    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") pod \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") "
Sep 17 14:12:47.530472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530415    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca\") pod \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\" (UID: \"3e227bf1-6855-4a34-9a2f-bf95ee3f4815\") "
Sep 17 14:12:47.530472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530461    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7x5h\" (UniqueName: \"kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h\") pod \"3c4ace8e-bb47-4aae-8584-804d82778410\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") "
Sep 17 14:12:47.530647 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530506    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume\") pod \"3c4ace8e-bb47-4aae-8584-804d82778410\" (UID: \"3c4ace8e-bb47-4aae-8584-804d82778410\") "
Sep 17 14:12:47.531104 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.530990    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c4ace8e-bb47-4aae-8584-804d82778410" (UID: "3c4ace8e-bb47-4aae-8584-804d82778410"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:12:47.531380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.531178    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e227bf1-6855-4a34-9a2f-bf95ee3f4815" (UID: "3e227bf1-6855-4a34-9a2f-bf95ee3f4815"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:12:47.532802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.532764    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h" (OuterVolumeSpecName: "kube-api-access-z7x5h") pod "3c4ace8e-bb47-4aae-8584-804d82778410" (UID: "3c4ace8e-bb47-4aae-8584-804d82778410"). InnerVolumeSpecName "kube-api-access-z7x5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:12:47.532802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.532789    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3e227bf1-6855-4a34-9a2f-bf95ee3f4815" (UID: "3e227bf1-6855-4a34-9a2f-bf95ee3f4815"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:12:47.533020 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.532997    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e227bf1-6855-4a34-9a2f-bf95ee3f4815" (UID: "3e227bf1-6855-4a34-9a2f-bf95ee3f4815"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:12:47.533109 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.533053    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx" (OuterVolumeSpecName: "kube-api-access-njtfx") pod "3e227bf1-6855-4a34-9a2f-bf95ee3f4815" (UID: "3e227bf1-6855-4a34-9a2f-bf95ee3f4815"). InnerVolumeSpecName "kube-api-access-njtfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631581    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631613    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7x5h\" (UniqueName: \"kubernetes.io/projected/3c4ace8e-bb47-4aae-8584-804d82778410-kube-api-access-z7x5h\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631623    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c4ace8e-bb47-4aae-8584-804d82778410-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631631    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njtfx\" (UniqueName: \"kubernetes.io/projected/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-kube-api-access-njtfx\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631641    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.631666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.631649    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e227bf1-6855-4a34-9a2f-bf95ee3f4815-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:12:47.793063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.793035    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:47.797988 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.797968    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-dd410a86-66998bb957-z84cc"]
Sep 17 14:12:47.808748 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.808705    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:47.810314 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:47.810297    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-dd410a86-647796f5c8-swlr9"]
Sep 17 14:12:48.182545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:48.182513    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c4ace8e-bb47-4aae-8584-804d82778410" path="/var/lib/kubelet/pods/3c4ace8e-bb47-4aae-8584-804d82778410/volumes"
Sep 17 14:12:48.182944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:48.182930    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" path="/var/lib/kubelet/pods/3e227bf1-6855-4a34-9a2f-bf95ee3f4815/volumes"
Sep 17 14:12:54.163400 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.163365    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:12:54.164033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164013    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c4ace8e-bb47-4aae-8584-804d82778410" containerName="payload-processing"
Sep 17 14:12:54.164085 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164038    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c4ace8e-bb47-4aae-8584-804d82778410" containerName="payload-processing"
Sep 17 14:12:54.164085 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164053    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" containerName="maas-api"
Sep 17 14:12:54.164085 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164061    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" containerName="maas-api"
Sep 17 14:12:54.164178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164160    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c4ace8e-bb47-4aae-8584-804d82778410" containerName="payload-processing"
Sep 17 14:12:54.164178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.164175    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e227bf1-6855-4a34-9a2f-bf95ee3f4815" containerName="maas-api"
Sep 17 14:12:54.166898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.166880    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.169836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.169812    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1c07ea69-serving-cert\""
Sep 17 14:12:54.183459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.183435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:12:54.296941 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.296905    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.297153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.296960    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.297225 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.297174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqmcs\" (UniqueName: \"kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.297280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.297219    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.398465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.398435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.398640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.398484    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.398640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.398547    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqmcs\" (UniqueName: \"kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.398640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.398569    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.399120 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.399099    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.401242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.401211    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.401360 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.401270    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.408138 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.408118    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqmcs\" (UniqueName: \"kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs\") pod \"maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.479977 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.479909    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:54.762796 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.762707    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:12:54.764843 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:12:54.764812    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0757c298_e0a2_4e02_9a4e_614aec0fd84d.slice/crio-0f6dcee3a59f1e1d42c02f74cb4e73e21056fc242c6cefbb9e117ce641dc8c55 WatchSource:0}: Error finding container 0f6dcee3a59f1e1d42c02f74cb4e73e21056fc242c6cefbb9e117ce641dc8c55: Status 404 returned error can't find the container with id 0f6dcee3a59f1e1d42c02f74cb4e73e21056fc242c6cefbb9e117ce641dc8c55
Sep 17 14:12:54.834338 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.834304    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:12:54.837375 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.837355    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:54.840116 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.840091    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1c07ea69\""
Sep 17 14:12:54.840253 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.840102    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1c07ea69-dockercfg-wsbd5\""
Sep 17 14:12:54.848141 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.848105    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:12:54.903688 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.903652    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcspn\" (UniqueName: \"kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:54.903862 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:54.903705    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.005333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.005291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcspn\" (UniqueName: \"kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.005519 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.005339    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.006144 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.006117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.013986 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.013938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcspn\" (UniqueName: \"kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn\") pod \"payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") " pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.150328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.150293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.425711 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.425685    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:12:55.512970 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.512941    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" event={"ID":"4089f0e0-42ee-4915-8a7b-eded752f0e1e","Type":"ContainerStarted","Data":"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"}
Sep 17 14:12:55.513427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.513033    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:12:55.513427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.513059    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" event={"ID":"4089f0e0-42ee-4915-8a7b-eded752f0e1e","Type":"ContainerStarted","Data":"36174aaeb8ab656ea58ecd683da8c58e44fe920e1f9466c2e3867b896365e0b0"}
Sep 17 14:12:55.514584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.514563    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" event={"ID":"0757c298-e0a2-4e02-9a4e-614aec0fd84d","Type":"ContainerStarted","Data":"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"}
Sep 17 14:12:55.514669 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.514605    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" event={"ID":"0757c298-e0a2-4e02-9a4e-614aec0fd84d","Type":"ContainerStarted","Data":"0f6dcee3a59f1e1d42c02f74cb4e73e21056fc242c6cefbb9e117ce641dc8c55"}
Sep 17 14:12:55.514718 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.514684    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:12:55.536826 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.536722    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" podStartSLOduration=1.53670818 podStartE2EDuration="1.53670818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:55.535089668 +0000 UTC m=+1017.771967144" watchObservedRunningTime="2026-09-17 14:12:55.53670818 +0000 UTC m=+1017.773585657"
Sep 17 14:12:55.557498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.557441    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" podStartSLOduration=1.102348946 podStartE2EDuration="1.557419306s" totalImagesPullingTime="455.07036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:12:55.554470403 +0000 UTC m=+1017.791347879" watchObservedRunningTime="2026-09-17 14:12:55.557419306 +0000 UTC m=+1017.794296790"
Sep 17 14:12:55.609378 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:12:55.609336    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:13:01.526773 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:01.526723    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:13:06.523407 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:06.523377    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:13:13.715931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:13.715884    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:13:24.159709 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.159672    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:13:24.160701 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.160666    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" podUID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" containerName="maas-api" containerID="cri-o://5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d" gracePeriod=30
Sep 17 14:13:24.244242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.244207    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:13:24.244478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.244448    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" podUID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" containerName="payload-processing" containerID="cri-o://ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69" gracePeriod=30
Sep 17 14:13:24.427254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.427223    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:13:24.484364 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.484335    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca\") pod \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") "
Sep 17 14:13:24.484527 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.484407    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls\") pod \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") "
Sep 17 14:13:24.484527 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.484464    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs\") pod \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") "
Sep 17 14:13:24.484527 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.484496    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqmcs\" (UniqueName: \"kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs\") pod \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\" (UID: \"0757c298-e0a2-4e02-9a4e-614aec0fd84d\") "
Sep 17 14:13:24.484758 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.484713    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0757c298-e0a2-4e02-9a4e-614aec0fd84d" (UID: "0757c298-e0a2-4e02-9a4e-614aec0fd84d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:13:24.485039 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.485010    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0757c298-e0a2-4e02-9a4e-614aec0fd84d-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.487264 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.487222    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0757c298-e0a2-4e02-9a4e-614aec0fd84d" (UID: "0757c298-e0a2-4e02-9a4e-614aec0fd84d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:13:24.487371 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.487296    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0757c298-e0a2-4e02-9a4e-614aec0fd84d" (UID: "0757c298-e0a2-4e02-9a4e-614aec0fd84d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:13:24.487371 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.487324    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs" (OuterVolumeSpecName: "kube-api-access-mqmcs") pod "0757c298-e0a2-4e02-9a4e-614aec0fd84d" (UID: "0757c298-e0a2-4e02-9a4e-614aec0fd84d"). InnerVolumeSpecName "kube-api-access-mqmcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:13:24.502691 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.502673    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:13:24.585519 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585461    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcspn\" (UniqueName: \"kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn\") pod \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") "
Sep 17 14:13:24.585519 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585529    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume\") pod \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\" (UID: \"4089f0e0-42ee-4915-8a7b-eded752f0e1e\") "
Sep 17 14:13:24.585876 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585859    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.585947 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585881    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0757c298-e0a2-4e02-9a4e-614aec0fd84d-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.585947 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585896    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqmcs\" (UniqueName: \"kubernetes.io/projected/0757c298-e0a2-4e02-9a4e-614aec0fd84d-kube-api-access-mqmcs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.586043 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.585991    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4089f0e0-42ee-4915-8a7b-eded752f0e1e" (UID: "4089f0e0-42ee-4915-8a7b-eded752f0e1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:13:24.588529 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.588495    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn" (OuterVolumeSpecName: "kube-api-access-fcspn") pod "4089f0e0-42ee-4915-8a7b-eded752f0e1e" (UID: "4089f0e0-42ee-4915-8a7b-eded752f0e1e"). InnerVolumeSpecName "kube-api-access-fcspn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:13:24.674145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.674060    2570 generic.go:356] "Generic (PLEG): container finished" podID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" containerID="ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69" exitCode=0
Sep 17 14:13:24.674145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.674103    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"
Sep 17 14:13:24.674345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.674150    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" event={"ID":"4089f0e0-42ee-4915-8a7b-eded752f0e1e","Type":"ContainerDied","Data":"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"}
Sep 17 14:13:24.674345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.674196    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm" event={"ID":"4089f0e0-42ee-4915-8a7b-eded752f0e1e","Type":"ContainerDied","Data":"36174aaeb8ab656ea58ecd683da8c58e44fe920e1f9466c2e3867b896365e0b0"}
Sep 17 14:13:24.674345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.674226    2570 scope.go:117] "RemoveContainer" containerID="ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"
Sep 17 14:13:24.675576 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.675550    2570 generic.go:356] "Generic (PLEG): container finished" podID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" containerID="5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d" exitCode=0
Sep 17 14:13:24.675712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.675579    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" event={"ID":"0757c298-e0a2-4e02-9a4e-614aec0fd84d","Type":"ContainerDied","Data":"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"}
Sep 17 14:13:24.675712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.675605    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"
Sep 17 14:13:24.675712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.675630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf" event={"ID":"0757c298-e0a2-4e02-9a4e-614aec0fd84d","Type":"ContainerDied","Data":"0f6dcee3a59f1e1d42c02f74cb4e73e21056fc242c6cefbb9e117ce641dc8c55"}
Sep 17 14:13:24.683583 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.683567    2570 scope.go:117] "RemoveContainer" containerID="ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"
Sep 17 14:13:24.683903 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:13:24.683881    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69\": container with ID starting with ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69 not found: ID does not exist" containerID="ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"
Sep 17 14:13:24.684003 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.683912    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69"} err="failed to get container status \"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69\": rpc error: code = NotFound desc = could not find container \"ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69\": container with ID starting with ab569607a79614d5423440c035c7317d7b6eeae14b31aabb99941c01f0548d69 not found: ID does not exist"
Sep 17 14:13:24.684003 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.683937    2570 scope.go:117] "RemoveContainer" containerID="5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"
Sep 17 14:13:24.686582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.686561    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcspn\" (UniqueName: \"kubernetes.io/projected/4089f0e0-42ee-4915-8a7b-eded752f0e1e-kube-api-access-fcspn\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.686582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.686579    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4089f0e0-42ee-4915-8a7b-eded752f0e1e-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:13:24.692241 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.692222    2570 scope.go:117] "RemoveContainer" containerID="5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"
Sep 17 14:13:24.692490 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:13:24.692471    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d\": container with ID starting with 5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d not found: ID does not exist" containerID="5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"
Sep 17 14:13:24.692546 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.692498    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d"} err="failed to get container status \"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d\": rpc error: code = NotFound desc = could not find container \"5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d\": container with ID starting with 5bf629652f2350cd8463ddbd2fee8d6fb9e85f6ffc099d7373cc4e22ea0eae1d not found: ID does not exist"
Sep 17 14:13:24.698769 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.698750    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:13:24.703012 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.702994    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1c07ea69-7cb44d7488-sqrbf"]
Sep 17 14:13:24.716153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.716131    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:13:24.720009 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:24.719987    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1c07ea69-7db94d78c9-fksvm"]
Sep 17 14:13:26.181999 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:26.181965    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" path="/var/lib/kubelet/pods/0757c298-e0a2-4e02-9a4e-614aec0fd84d/volumes"
Sep 17 14:13:26.182359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:26.182342    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" path="/var/lib/kubelet/pods/4089f0e0-42ee-4915-8a7b-eded752f0e1e/volumes"
Sep 17 14:13:30.327588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.327555    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:13:30.328212 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328192    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" containerName="maas-api"
Sep 17 14:13:30.328266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328218    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" containerName="maas-api"
Sep 17 14:13:30.328266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328246    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" containerName="payload-processing"
Sep 17 14:13:30.328266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328255    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" containerName="payload-processing"
Sep 17 14:13:30.328359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328347    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0757c298-e0a2-4e02-9a4e-614aec0fd84d" containerName="maas-api"
Sep 17 14:13:30.328392 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.328368    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4089f0e0-42ee-4915-8a7b-eded752f0e1e" containerName="payload-processing"
Sep 17 14:13:30.332006 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.331985    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.334686 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.334665    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-9af6d23e-serving-cert\""
Sep 17 14:13:30.340409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.340040    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:13:30.438682 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.438503    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.438682 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.438551    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.438682 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.438596    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.438682 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.438637    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk48p\" (UniqueName: \"kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.539937 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.539906    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.539937 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.539939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.540198 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.539965    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.540198 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:13:30.540073    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-serving-cert: secret "maas-api-e2e-derive-9af6d23e-serving-cert" not found
Sep 17 14:13:30.540198 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.540089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk48p\" (UniqueName: \"kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.540198 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:13:30.540147    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls podName:a6ad7ced-ebda-4d82-a440-827a60ea50c1 nodeName:}" failed. No retries permitted until 2026-09-17 14:13:31.040127621 +0000 UTC m=+1053.277005076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls") pod "maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" (UID: "a6ad7ced-ebda-4d82-a440-827a60ea50c1") : secret "maas-api-e2e-derive-9af6d23e-serving-cert" not found
Sep 17 14:13:30.540659 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.540638    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.542620 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.542601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.553578 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.553549    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk48p\" (UniqueName: \"kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:30.827912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:30.827879    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:13:31.045510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:31.045475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:31.048148 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:31.048117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") pod \"maas-api-e2e-derive-9af6d23e-75f46765cc-t684s\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:31.244148 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:31.244112    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:31.523671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:31.523643    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:13:31.525838 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:13:31.525805    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ad7ced_ebda_4d82_a440_827a60ea50c1.slice/crio-f2b1b17a473c6c9e0bb742aacb6443e49ed57bd12f6808f643dce8c436e355c8 WatchSource:0}: Error finding container f2b1b17a473c6c9e0bb742aacb6443e49ed57bd12f6808f643dce8c436e355c8: Status 404 returned error can't find the container with id f2b1b17a473c6c9e0bb742aacb6443e49ed57bd12f6808f643dce8c436e355c8
Sep 17 14:13:31.715271 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:31.715238    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" event={"ID":"a6ad7ced-ebda-4d82-a440-827a60ea50c1","Type":"ContainerStarted","Data":"f2b1b17a473c6c9e0bb742aacb6443e49ed57bd12f6808f643dce8c436e355c8"}
Sep 17 14:13:32.722331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:32.722291    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" event={"ID":"a6ad7ced-ebda-4d82-a440-827a60ea50c1","Type":"ContainerStarted","Data":"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"}
Sep 17 14:13:32.722778 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:32.722479    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:38.732847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:38.732816    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:13:38.756757 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:38.756053    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" podStartSLOduration=8.340728143 podStartE2EDuration="8.756024602s" totalImagesPullingTime="415.296459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:13:32.746076673 +0000 UTC m=+1054.982954149" watchObservedRunningTime="2026-09-17 14:13:38.756024602 +0000 UTC m=+1060.992902161"
Sep 17 14:13:50.924261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:13:50.924177    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:14:01.414632 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.414598    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:14:01.415095 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.414857    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" podUID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" containerName="payload-pre-processing" containerID="cri-o://ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987" gracePeriod=30
Sep 17 14:14:01.679647 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.679583    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:14:01.739420 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.739385    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5qnl\" (UniqueName: \"kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl\") pod \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") "
Sep 17 14:14:01.739592 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.739425    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume\") pod \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\" (UID: \"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8\") "
Sep 17 14:14:01.739835 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.739810    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" (UID: "54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:14:01.741864 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.741778    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl" (OuterVolumeSpecName: "kube-api-access-n5qnl") pod "54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" (UID: "54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8"). InnerVolumeSpecName "kube-api-access-n5qnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:01.840201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.840166    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5qnl\" (UniqueName: \"kubernetes.io/projected/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-kube-api-access-n5qnl\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:01.840201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.840195    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:01.889691 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.889658    2570 generic.go:356] "Generic (PLEG): container finished" podID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" containerID="ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987" exitCode=0
Sep 17 14:14:01.889891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.889704    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"
Sep 17 14:14:01.889891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.889708    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" event={"ID":"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8","Type":"ContainerDied","Data":"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"}
Sep 17 14:14:01.889891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.889777    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp" event={"ID":"54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8","Type":"ContainerDied","Data":"83e4e060cca23a55f3c2ae89b8a172611ff747ce0227b1a9a28844a2723ee8b6"}
Sep 17 14:14:01.889891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.889806    2570 scope.go:117] "RemoveContainer" containerID="ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"
Sep 17 14:14:01.898955 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.898937    2570 scope.go:117] "RemoveContainer" containerID="ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"
Sep 17 14:14:01.899217 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:01.899200    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987\": container with ID starting with ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987 not found: ID does not exist" containerID="ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"
Sep 17 14:14:01.899288 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.899225    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987"} err="failed to get container status \"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987\": rpc error: code = NotFound desc = could not find container \"ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987\": container with ID starting with ca4391c3d82faa9d2abdf241f779c2ecdc6bdcddf026510cfdcb7f805f7d5987 not found: ID does not exist"
Sep 17 14:14:01.912125 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.912098    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:14:01.916112 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:01.916092    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0458f0-5b596df974-k8rqp"]
Sep 17 14:14:02.182927 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:02.182897    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" path="/var/lib/kubelet/pods/54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8/volumes"
Sep 17 14:14:06.246803 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:06.246769    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:14:07.528411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.528379    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:14:07.528894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.528876    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" containerName="payload-pre-processing"
Sep 17 14:14:07.528952 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.528898    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" containerName="payload-pre-processing"
Sep 17 14:14:07.529046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.529034    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="54bdcf0e-2bcc-4ea0-bda9-cacf24547ab8" containerName="payload-pre-processing"
Sep 17 14:14:07.533388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.533371    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.536051 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.536026    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-96cfc6\"/\"openshift-service-ca.crt\""
Sep 17 14:14:07.536191 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.536028    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-96cfc6\"/\"default-dockercfg-s6brf\""
Sep 17 14:14:07.537252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.537238    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-96cfc6\"/\"kube-root-ca.crt\""
Sep 17 14:14:07.537313 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.537236    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-96cfc6\"/\"test-model-96cfc6-kserve-self-signed-certs\""
Sep 17 14:14:07.542872 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.542851    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:14:07.599678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.599859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599692    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kd9r\" (UniqueName: \"kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.599859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.599859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599797    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.599859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.599859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.599851    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700497 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700460    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kd9r\" (UniqueName: \"kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700591    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.700707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.701033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.700974    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.701033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.701022    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.701105 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.701049    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.703080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.703063    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.703196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.703179    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.711042 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.711014    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kd9r\" (UniqueName: \"kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r\") pod \"test-model-96cfc6-kserve-79d4749499-ntkrr\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") " pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:07.844098 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:07.844007    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:08.143464 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:08.143442    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:14:08.145688 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:14:08.145664    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78bb0f84_029b_48eb_92fa_6a638eeb63bb.slice/crio-9e845aff21d9d05020d44621e508e1eb4a902119254560b9521218b7984275df WatchSource:0}: Error finding container 9e845aff21d9d05020d44621e508e1eb4a902119254560b9521218b7984275df: Status 404 returned error can't find the container with id 9e845aff21d9d05020d44621e508e1eb4a902119254560b9521218b7984275df
Sep 17 14:14:08.929066 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:08.929028    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" event={"ID":"78bb0f84-029b-48eb-92fa-6a638eeb63bb","Type":"ContainerStarted","Data":"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"}
Sep 17 14:14:08.929430 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:08.929077    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" event={"ID":"78bb0f84-029b-48eb-92fa-6a638eeb63bb","Type":"ContainerStarted","Data":"9e845aff21d9d05020d44621e508e1eb4a902119254560b9521218b7984275df"}
Sep 17 14:14:08.950317 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:08.950263    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" podStartSLOduration=1.95024164 podStartE2EDuration="1.95024164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:08.948485707 +0000 UTC m=+1091.185363184" watchObservedRunningTime="2026-09-17 14:14:08.95024164 +0000 UTC m=+1091.187119118"
Sep 17 14:14:10.441133 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.441099    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:14:10.441570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.441338    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" podUID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" containerName="maas-api" containerID="cri-o://5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a" gracePeriod=30
Sep 17 14:14:10.702511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.702452    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:14:10.832831 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.832800    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca\") pod \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") "
Sep 17 14:14:10.833000 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.832887    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") pod \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") "
Sep 17 14:14:10.833000 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.832950    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs\") pod \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") "
Sep 17 14:14:10.833000 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.832975    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jk48p\" (UniqueName: \"kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p\") pod \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\" (UID: \"a6ad7ced-ebda-4d82-a440-827a60ea50c1\") "
Sep 17 14:14:10.833210 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.833187    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6ad7ced-ebda-4d82-a440-827a60ea50c1" (UID: "a6ad7ced-ebda-4d82-a440-827a60ea50c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:14:10.833358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.833340    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ad7ced-ebda-4d82-a440-827a60ea50c1-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:10.835153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.835119    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p" (OuterVolumeSpecName: "kube-api-access-jk48p") pod "a6ad7ced-ebda-4d82-a440-827a60ea50c1" (UID: "a6ad7ced-ebda-4d82-a440-827a60ea50c1"). InnerVolumeSpecName "kube-api-access-jk48p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:10.835267 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.835167    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6ad7ced-ebda-4d82-a440-827a60ea50c1" (UID: "a6ad7ced-ebda-4d82-a440-827a60ea50c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:14:10.835267 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.835207    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a6ad7ced-ebda-4d82-a440-827a60ea50c1" (UID: "a6ad7ced-ebda-4d82-a440-827a60ea50c1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:14:10.934378 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.934347    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:10.934378 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.934380    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk48p\" (UniqueName: \"kubernetes.io/projected/a6ad7ced-ebda-4d82-a440-827a60ea50c1-kube-api-access-jk48p\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:10.934597 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.934396    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ad7ced-ebda-4d82-a440-827a60ea50c1-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:10.941833 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.941795    2570 generic.go:356] "Generic (PLEG): container finished" podID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" containerID="5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a" exitCode=0
Sep 17 14:14:10.941978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.941848    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" event={"ID":"a6ad7ced-ebda-4d82-a440-827a60ea50c1","Type":"ContainerDied","Data":"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"}
Sep 17 14:14:10.941978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.941890    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s" event={"ID":"a6ad7ced-ebda-4d82-a440-827a60ea50c1","Type":"ContainerDied","Data":"f2b1b17a473c6c9e0bb742aacb6443e49ed57bd12f6808f643dce8c436e355c8"}
Sep 17 14:14:10.941978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.941912    2570 scope.go:117] "RemoveContainer" containerID="5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"
Sep 17 14:14:10.941978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.941852    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"
Sep 17 14:14:10.953933 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.953913    2570 scope.go:117] "RemoveContainer" containerID="5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"
Sep 17 14:14:10.954333 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:10.954308    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a\": container with ID starting with 5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a not found: ID does not exist" containerID="5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"
Sep 17 14:14:10.954433 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.954349    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a"} err="failed to get container status \"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a\": rpc error: code = NotFound desc = could not find container \"5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a\": container with ID starting with 5549e07af770b4ae7f5d621c0a363388f11a7a36cd4142c58e20d5a7efa2348a not found: ID does not exist"
Sep 17 14:14:10.969542 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.969518    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:14:10.973944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:10.973921    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-9af6d23e-75f46765cc-t684s"]
Sep 17 14:14:12.182081 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:12.182049    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" path="/var/lib/kubelet/pods/a6ad7ced-ebda-4d82-a440-827a60ea50c1/volumes"
Sep 17 14:14:17.844623 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:17.844584    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:17.845041 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:17.844641    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:17.860801 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:17.860778    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:17.990432 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:17.990409    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:14:28.626212 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:28.626180    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:14:45.286493 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:45.286461    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:14:45.287006 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:45.286709    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy" containerID="cri-o://fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea" gracePeriod=30
Sep 17 14:14:50.538051 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.538026    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:14:50.620111 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620018    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620111 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620063    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620120    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620139    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620181    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk5mq\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620222    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620282    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620322    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620360    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data\") pod \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\" (UID: \"da9ee91f-ff59-47a3-9549-9d2f12afeaf2\") "
Sep 17 14:14:50.620582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620356    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:50.620582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620398    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:50.620836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620694    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data" (OuterVolumeSpecName: "istio-data") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:50.620836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620784    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.620836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620807    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.620836 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620822    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.621024 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.620832    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:50.621069 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.621042    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:14:50.622537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.622504    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token" (OuterVolumeSpecName: "istio-token") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:50.622650 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.622530    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:14:50.623138 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.623104    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:14:50.623224 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.623139    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq" (OuterVolumeSpecName: "kube-api-access-wk5mq") pod "da9ee91f-ff59-47a3-9549-9d2f12afeaf2" (UID: "da9ee91f-ff59-47a3-9549-9d2f12afeaf2"). InnerVolumeSpecName "kube-api-access-wk5mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:14:50.721483 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721449    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.721483 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721480    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.721483 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721489    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.721715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721499    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk5mq\" (UniqueName: \"kubernetes.io/projected/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-kube-api-access-wk5mq\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.721715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721511    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:50.721715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:50.721520    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da9ee91f-ff59-47a3-9549-9d2f12afeaf2-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:14:51.158462 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.158424    2570 generic.go:356] "Generic (PLEG): container finished" podID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerID="fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea" exitCode=0
Sep 17 14:14:51.158668 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.158479    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerDied","Data":"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"}
Sep 17 14:14:51.158668 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.158494    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"
Sep 17 14:14:51.158668 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.158542    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g" event={"ID":"da9ee91f-ff59-47a3-9549-9d2f12afeaf2","Type":"ContainerDied","Data":"8bfdd67bc0cc04dcff0fc7349f9458e6661bb7059cc48187241d35ee9be6cf4c"}
Sep 17 14:14:51.158668 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.158569    2570 scope.go:117] "RemoveContainer" containerID="fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"
Sep 17 14:14:51.169095 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.169070    2570 scope.go:117] "RemoveContainer" containerID="6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"
Sep 17 14:14:51.177893 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.177875    2570 scope.go:117] "RemoveContainer" containerID="fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"
Sep 17 14:14:51.178169 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:51.178151    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea\": container with ID starting with fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea not found: ID does not exist" containerID="fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"
Sep 17 14:14:51.178218 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.178176    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea"} err="failed to get container status \"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea\": rpc error: code = NotFound desc = could not find container \"fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea\": container with ID starting with fd6fba3c2818b323c3628babfcafff89da37e0f4f3920ca14a8e83593d13c7ea not found: ID does not exist"
Sep 17 14:14:51.178218 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.178192    2570 scope.go:117] "RemoveContainer" containerID="6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"
Sep 17 14:14:51.178433 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:51.178413    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0\": container with ID starting with 6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0 not found: ID does not exist" containerID="6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"
Sep 17 14:14:51.178498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.178444    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0"} err="failed to get container status \"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0\": rpc error: code = NotFound desc = could not find container \"6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0\": container with ID starting with 6051b790815d05fe00aecce247c6482971e0477ae0b8054924d104a914721bf0 not found: ID does not exist"
Sep 17 14:14:51.186983 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.186964    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:14:51.194361 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:51.194341    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-0458f0-openshift-default-7b89fd88bd-55p5g"]
Sep 17 14:14:52.181834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:52.181797    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" path="/var/lib/kubelet/pods/da9ee91f-ff59-47a3-9549-9d2f12afeaf2/volumes"
Sep 17 14:14:52.618172 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:52.618072    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:14:53.939579 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.939549    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:14:53.940181 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940161    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:53.940223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940186    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:53.940223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940208    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" containerName="maas-api"
Sep 17 14:14:53.940223 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940216    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" containerName="maas-api"
Sep 17 14:14:53.940351 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940338    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:53.940388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940355    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6ad7ced-ebda-4d82-a440-827a60ea50c1" containerName="maas-api"
Sep 17 14:14:53.940388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.940367    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:53.945242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.945222    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:53.947793 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.947773    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-767170-serving-cert\""
Sep 17 14:14:53.956459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:53.956421    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:14:54.056945 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.055906    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqfl\" (UniqueName: \"kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.056945 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.056221    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.056945 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.056275    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.056945 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.056342    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.157946 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.157918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqfl\" (UniqueName: \"kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.158091 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.157954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.158091 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.157980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.158091 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.158029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.158203 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:54.158173    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-767170-serving-cert: secret "maas-api-e2e-explicit-767170-serving-cert" not found
Sep 17 14:14:54.158274 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:14:54.158261    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls podName:e1849fd6-7c97-4350-8fbe-f3ebae9ff412 nodeName:}" failed. No retries permitted until 2026-09-17 14:14:54.658237316 +0000 UTC m=+1136.895114789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls") pod "maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" (UID: "e1849fd6-7c97-4350-8fbe-f3ebae9ff412") : secret "maas-api-e2e-explicit-767170-serving-cert" not found
Sep 17 14:14:54.158771 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.158689    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.160571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.160553    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.175606 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.175586    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqfl\" (UniqueName: \"kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.647033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.646988    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:14:54.647415 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.647393    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:54.647465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.647420    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9ee91f-ff59-47a3-9549-9d2f12afeaf2" containerName="istio-proxy"
Sep 17 14:14:54.650687 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.650655    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.653263 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.653242    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-767170\""
Sep 17 14:14:54.661627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.661605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.664452 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.664427    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") pod \"maas-api-e2e-explicit-767170-77cc4b8b9b-t9295\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.665175 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.665156    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:14:54.762985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.762931    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7mpz\" (UniqueName: \"kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.763155 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.763025    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.858446 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.858410    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:54.863819 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.863791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7mpz\" (UniqueName: \"kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.863938 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.863860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.864488 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.864469    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.872363 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.872342    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7mpz\" (UniqueName: \"kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz\") pod \"payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:54.962212 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:54.962180    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:55.133282 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:55.133259    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:14:55.135588 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:14:55.135560    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1849fd6_7c97_4350_8fbe_f3ebae9ff412.slice/crio-b9ec7d43a7b7ed708abb45a90d2fd778e788340e1ad0fdac3a5c72e7a773472f WatchSource:0}: Error finding container b9ec7d43a7b7ed708abb45a90d2fd778e788340e1ad0fdac3a5c72e7a773472f: Status 404 returned error can't find the container with id b9ec7d43a7b7ed708abb45a90d2fd778e788340e1ad0fdac3a5c72e7a773472f
Sep 17 14:14:55.183098 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:55.183021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" event={"ID":"e1849fd6-7c97-4350-8fbe-f3ebae9ff412","Type":"ContainerStarted","Data":"b9ec7d43a7b7ed708abb45a90d2fd778e788340e1ad0fdac3a5c72e7a773472f"}
Sep 17 14:14:55.237585 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:55.237564    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:14:55.239802 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:14:55.239774    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda201efd4_5330_4827_983e_da814079e228.slice/crio-16586966d6c79e425b84ad73d2683cf43de87aec26f17d5763918d67059a934c WatchSource:0}: Error finding container 16586966d6c79e425b84ad73d2683cf43de87aec26f17d5763918d67059a934c: Status 404 returned error can't find the container with id 16586966d6c79e425b84ad73d2683cf43de87aec26f17d5763918d67059a934c
Sep 17 14:14:56.191185 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.191145    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" event={"ID":"e1849fd6-7c97-4350-8fbe-f3ebae9ff412","Type":"ContainerStarted","Data":"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"}
Sep 17 14:14:56.191625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.191254    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:14:56.192745 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.192710    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" event={"ID":"a201efd4-5330-4827-983e-da814079e228","Type":"ContainerStarted","Data":"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"}
Sep 17 14:14:56.192856 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.192770    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" event={"ID":"a201efd4-5330-4827-983e-da814079e228","Type":"ContainerStarted","Data":"16586966d6c79e425b84ad73d2683cf43de87aec26f17d5763918d67059a934c"}
Sep 17 14:14:56.192856 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.192829    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:14:56.213437 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.213391    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" podStartSLOduration=2.727056679 podStartE2EDuration="3.213376317s" totalImagesPullingTime="486.319638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:56.211927223 +0000 UTC m=+1138.448804702" watchObservedRunningTime="2026-09-17 14:14:56.213376317 +0000 UTC m=+1138.450253793"
Sep 17 14:14:56.243134 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:14:56.243088    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" podStartSLOduration=2.243072753 podStartE2EDuration="2.243072753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:14:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:14:56.241896652 +0000 UTC m=+1138.478774131" watchObservedRunningTime="2026-09-17 14:14:56.243072753 +0000 UTC m=+1138.479950229"
Sep 17 14:15:00.162994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.160344    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"]
Sep 17 14:15:00.166333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.166305    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"
Sep 17 14:15:00.169184 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.169156    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rp9ww\""
Sep 17 14:15:00.180536 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.180515    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"
Sep 17 14:15:00.185180 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.185157    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"]
Sep 17 14:15:00.194671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.194645    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"]
Sep 17 14:15:00.198616 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.198590    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"
Sep 17 14:15:00.206943 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.206916    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"]
Sep 17 14:15:00.220057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.220038    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"
Sep 17 14:15:00.469595 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.469566    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"]
Sep 17 14:15:00.470137 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:15:00.470113    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6c6844_e2e9_4b60_9e8a_63d293c2ae5c.slice/crio-753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744 WatchSource:0}: Error finding container 753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744: Status 404 returned error can't find the container with id 753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744
Sep 17 14:15:00.510595 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:00.510562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"]
Sep 17 14:15:00.511322 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:15:00.511301    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd536a4b9_9fa4_46fb_907b_5106890d0653.slice/crio-c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc WatchSource:0}: Error finding container c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc: Status 404 returned error can't find the container with id c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc
Sep 17 14:15:01.223186 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:01.223148    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4" event={"ID":"d536a4b9-9fa4-46fb-907b-5106890d0653","Type":"ContainerStarted","Data":"c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc"}
Sep 17 14:15:01.224319 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:01.224299    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz" event={"ID":"0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c","Type":"ContainerStarted","Data":"753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744"}
Sep 17 14:15:02.206606 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:02.206578    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:15:04.243466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:04.243429    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4" event={"ID":"d536a4b9-9fa4-46fb-907b-5106890d0653","Type":"ContainerStarted","Data":"4afe67a6c37b456fef690a414a03259bf5569e61eebbf929d452f876c7a07566"}
Sep 17 14:15:04.244972 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:04.244953    2570 generic.go:356] "Generic (PLEG): container finished" podID="0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c" containerID="c53ebb2a6444f3925c995d3252a91aea5ca8142851661adbffb4143f06d64682" exitCode=0
Sep 17 14:15:04.245088 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:04.244986    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz" event={"ID":"0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c","Type":"ContainerDied","Data":"c53ebb2a6444f3925c995d3252a91aea5ca8142851661adbffb4143f06d64682"}
Sep 17 14:15:04.262170 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:04.262120    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4" podStartSLOduration=1.720640098 podStartE2EDuration="4.262104821s" totalImagesPullingTime="2.541464723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:15:04.260868967 +0000 UTC m=+1146.497746445" watchObservedRunningTime="2026-09-17 14:15:04.262104821 +0000 UTC m=+1146.498982339"
Sep 17 14:15:05.251943 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:05.251903    2570 generic.go:356] "Generic (PLEG): container finished" podID="d536a4b9-9fa4-46fb-907b-5106890d0653" containerID="4afe67a6c37b456fef690a414a03259bf5569e61eebbf929d452f876c7a07566" exitCode=0
Sep 17 14:15:05.252441 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:05.252002    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4" event={"ID":"d536a4b9-9fa4-46fb-907b-5106890d0653","Type":"ContainerDied","Data":"4afe67a6c37b456fef690a414a03259bf5569e61eebbf929d452f876c7a07566"}
Sep 17 14:15:05.426570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:05.426543    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"
Sep 17 14:15:06.258802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:06.258769    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"
Sep 17 14:15:06.258802 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:06.258778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz" event={"ID":"0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c","Type":"ContainerDied","Data":"753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744"}
Sep 17 14:15:06.259264 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:06.258824    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="753b386c26cedf7efba18b00cdba946812945b3f52815e714fcddf7348507744"
Sep 17 14:15:06.411287 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:06.411259    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"
Sep 17 14:15:07.202845 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:07.202807    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:15:07.265648 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:07.265624    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"
Sep 17 14:15:07.266123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:07.265618    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4" event={"ID":"d536a4b9-9fa4-46fb-907b-5106890d0653","Type":"ContainerDied","Data":"c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc"}
Sep 17 14:15:07.266123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:07.265757    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2593cc9659bd102758219c898196a50185240589b71c3bf0a6e4cfc4d2f6ecc"
Sep 17 14:15:14.010345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:14.010307    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:15:16.916359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916324    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:15:16.916766 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916745    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d536a4b9-9fa4-46fb-907b-5106890d0653" containerName="cleanup"
Sep 17 14:15:16.916766 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916758    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d536a4b9-9fa4-46fb-907b-5106890d0653" containerName="cleanup"
Sep 17 14:15:16.916843 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916772    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c" containerName="cleanup"
Sep 17 14:15:16.916843 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916778    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c" containerName="cleanup"
Sep 17 14:15:16.916903 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916846    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d536a4b9-9fa4-46fb-907b-5106890d0653" containerName="cleanup"
Sep 17 14:15:16.916903 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.916855    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c" containerName="cleanup"
Sep 17 14:15:16.920068 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.920050    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.922926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.922906    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-079b38\"/\"premium-w4-079b38-kserve-self-signed-certs\""
Sep 17 14:15:16.931519 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.931494    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:15:16.980427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980396    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.980427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980433    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.980698 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.980698 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980547    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.980698 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980655    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt6w5\" (UniqueName: \"kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:16.980875 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:16.980721    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081604    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt6w5\" (UniqueName: \"kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081777    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081807    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.081923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.081866    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.082307 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.082283    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.082570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.082543    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.082985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.082961    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.085077 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.085048    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.085332 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.085308    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.095291 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.095260    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt6w5\" (UniqueName: \"kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5\") pod \"premium-w4-079b38-kserve-7cc97d947-p5gvd\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") " pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.228898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.228816    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:17.519944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:17.519869    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:15:17.523666 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:15:17.523623    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6afa62d0_95db_4615_a6bf_ab2bc5b5eee7.slice/crio-232bd8255581862c55730e808cf0077f4dde50ec3bc5b7fe1f71f151f9ef9900 WatchSource:0}: Error finding container 232bd8255581862c55730e808cf0077f4dde50ec3bc5b7fe1f71f151f9ef9900: Status 404 returned error can't find the container with id 232bd8255581862c55730e808cf0077f4dde50ec3bc5b7fe1f71f151f9ef9900
Sep 17 14:15:18.325913 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:18.325878    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" event={"ID":"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7","Type":"ContainerStarted","Data":"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"}
Sep 17 14:15:18.326267 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:18.325923    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" event={"ID":"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7","Type":"ContainerStarted","Data":"232bd8255581862c55730e808cf0077f4dde50ec3bc5b7fe1f71f151f9ef9900"}
Sep 17 14:15:18.356382 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:18.356335    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" podStartSLOduration=2.356318061 podStartE2EDuration="2.356318061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:15:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:15:18.356179128 +0000 UTC m=+1160.593056604" watchObservedRunningTime="2026-09-17 14:15:18.356318061 +0000 UTC m=+1160.593195583"
Sep 17 14:15:27.229418 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:27.229372    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:27.229915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:27.229441    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:27.242316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:27.242295    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:27.386921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:27.386892    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:15:35.212492 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:15:35.212460    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:16:06.130075 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:06.130037    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:16:09.157917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.157885    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:09.161431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.161415    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.164817 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.164799    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-4a43447e-dockercfg-4mlls\""
Sep 17 14:16:09.164921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.164834    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-4a43447e\""
Sep 17 14:16:09.171782 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.171759    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:09.303869 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.303823    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.304058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.303956    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clx99\" (UniqueName: \"kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.404692 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.404653    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clx99\" (UniqueName: \"kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.404886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.404708    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.405342 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.405324    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.418415 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.418353    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clx99\" (UniqueName: \"kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99\") pod \"payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") " pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.472286 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.472246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:09.760273 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:09.760250    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:09.762214 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:16:09.762187    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8008639b_4ee0_481c_9781_44458b721e06.slice/crio-66361d055b43e0db6d23ed6395868875fd638a67b6009b278bd9a0c91d92214f WatchSource:0}: Error finding container 66361d055b43e0db6d23ed6395868875fd638a67b6009b278bd9a0c91d92214f: Status 404 returned error can't find the container with id 66361d055b43e0db6d23ed6395868875fd638a67b6009b278bd9a0c91d92214f
Sep 17 14:16:10.606813 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:10.606773    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" event={"ID":"8008639b-4ee0-481c-9781-44458b721e06","Type":"ContainerStarted","Data":"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"}
Sep 17 14:16:10.607188 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:10.606836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" event={"ID":"8008639b-4ee0-481c-9781-44458b721e06","Type":"ContainerStarted","Data":"66361d055b43e0db6d23ed6395868875fd638a67b6009b278bd9a0c91d92214f"}
Sep 17 14:16:10.607188 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:10.606952    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:10.634374 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:10.634334    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" podStartSLOduration=1.634320099 podStartE2EDuration="1.634320099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:16:10.632863949 +0000 UTC m=+1212.869741425" watchObservedRunningTime="2026-09-17 14:16:10.634320099 +0000 UTC m=+1212.871197575"
Sep 17 14:16:21.614745 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:21.614711    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:36.121269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:36.121234    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:16:51.942149 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:51.941944    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:51.943126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:51.943064    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" podUID="8008639b-4ee0-481c-9781-44458b721e06" containerName="payload-processing" containerID="cri-o://889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0" gracePeriod=30
Sep 17 14:16:52.210013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.209991    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:52.325220 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.325195    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume\") pod \"8008639b-4ee0-481c-9781-44458b721e06\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") "
Sep 17 14:16:52.325387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.325319    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clx99\" (UniqueName: \"kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99\") pod \"8008639b-4ee0-481c-9781-44458b721e06\" (UID: \"8008639b-4ee0-481c-9781-44458b721e06\") "
Sep 17 14:16:52.325610 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.325581    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8008639b-4ee0-481c-9781-44458b721e06" (UID: "8008639b-4ee0-481c-9781-44458b721e06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:16:52.327555 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.327525    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99" (OuterVolumeSpecName: "kube-api-access-clx99") pod "8008639b-4ee0-481c-9781-44458b721e06" (UID: "8008639b-4ee0-481c-9781-44458b721e06"). InnerVolumeSpecName "kube-api-access-clx99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:16:52.426280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.426249    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clx99\" (UniqueName: \"kubernetes.io/projected/8008639b-4ee0-481c-9781-44458b721e06-kube-api-access-clx99\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:16:52.426280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.426275    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8008639b-4ee0-481c-9781-44458b721e06-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:16:52.847319 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.847288    2570 generic.go:356] "Generic (PLEG): container finished" podID="8008639b-4ee0-481c-9781-44458b721e06" containerID="889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0" exitCode=0
Sep 17 14:16:52.847510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.847342    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"
Sep 17 14:16:52.847510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.847365    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" event={"ID":"8008639b-4ee0-481c-9781-44458b721e06","Type":"ContainerDied","Data":"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"}
Sep 17 14:16:52.847510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.847415    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs" event={"ID":"8008639b-4ee0-481c-9781-44458b721e06","Type":"ContainerDied","Data":"66361d055b43e0db6d23ed6395868875fd638a67b6009b278bd9a0c91d92214f"}
Sep 17 14:16:52.847510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.847435    2570 scope.go:117] "RemoveContainer" containerID="889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"
Sep 17 14:16:52.857692 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.857674    2570 scope.go:117] "RemoveContainer" containerID="889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"
Sep 17 14:16:52.858027 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:16:52.857999    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0\": container with ID starting with 889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0 not found: ID does not exist" containerID="889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"
Sep 17 14:16:52.858093 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.858036    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0"} err="failed to get container status \"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0\": rpc error: code = NotFound desc = could not find container \"889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0\": container with ID starting with 889890e09d4ddc5d54282d970156c1df0c3d07622e2b1051c5cac87b6102a4e0 not found: ID does not exist"
Sep 17 14:16:52.875822 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.875801    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:52.877824 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:52.877805    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-4a43447e-6d6574f785-5r4bs"]
Sep 17 14:16:54.182119 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:54.182085    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8008639b-4ee0-481c-9781-44458b721e06" path="/var/lib/kubelet/pods/8008639b-4ee0-481c-9781-44458b721e06/volumes"
Sep 17 14:16:59.219467 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:16:59.219428    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:17:03.391437 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.391405    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:17:03.391939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.391590    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" podUID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" containerName="maas-api" containerID="cri-o://3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224" gracePeriod=30
Sep 17 14:17:03.526954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.526921    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"]
Sep 17 14:17:03.543896 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.543866    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-767170-29827575-grhlz"]
Sep 17 14:17:03.653925 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.653895    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:17:03.660293 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.660251    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:17:03.660505 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.660464    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" podUID="a201efd4-5330-4827-983e-da814079e228" containerName="payload-pre-processing" containerID="cri-o://1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d" gracePeriod=30
Sep 17 14:17:03.840191 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.840164    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtqfl\" (UniqueName: \"kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl\") pod \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") "
Sep 17 14:17:03.840341 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.840246    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca\") pod \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") "
Sep 17 14:17:03.840596 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.840569    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e1849fd6-7c97-4350-8fbe-f3ebae9ff412" (UID: "e1849fd6-7c97-4350-8fbe-f3ebae9ff412"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:17:03.840713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.840672    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") pod \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") "
Sep 17 14:17:03.840806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.840762    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs\") pod \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\" (UID: \"e1849fd6-7c97-4350-8fbe-f3ebae9ff412\") "
Sep 17 14:17:03.841150 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.841125    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:03.843179 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.843142    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl" (OuterVolumeSpecName: "kube-api-access-dtqfl") pod "e1849fd6-7c97-4350-8fbe-f3ebae9ff412" (UID: "e1849fd6-7c97-4350-8fbe-f3ebae9ff412"). InnerVolumeSpecName "kube-api-access-dtqfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:17:03.844781 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.844712    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e1849fd6-7c97-4350-8fbe-f3ebae9ff412" (UID: "e1849fd6-7c97-4350-8fbe-f3ebae9ff412"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:17:03.845967 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.845944    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e1849fd6-7c97-4350-8fbe-f3ebae9ff412" (UID: "e1849fd6-7c97-4350-8fbe-f3ebae9ff412"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:17:03.913279 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.913249    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:17:03.922108 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.922010    2570 generic.go:356] "Generic (PLEG): container finished" podID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" containerID="3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224" exitCode=0
Sep 17 14:17:03.922108 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.922063    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"
Sep 17 14:17:03.922108 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.922065    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" event={"ID":"e1849fd6-7c97-4350-8fbe-f3ebae9ff412","Type":"ContainerDied","Data":"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"}
Sep 17 14:17:03.922354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.922137    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295" event={"ID":"e1849fd6-7c97-4350-8fbe-f3ebae9ff412","Type":"ContainerDied","Data":"b9ec7d43a7b7ed708abb45a90d2fd778e788340e1ad0fdac3a5c72e7a773472f"}
Sep 17 14:17:03.922354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.922162    2570 scope.go:117] "RemoveContainer" containerID="3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"
Sep 17 14:17:03.925817 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.925797    2570 generic.go:356] "Generic (PLEG): container finished" podID="a201efd4-5330-4827-983e-da814079e228" containerID="1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d" exitCode=0
Sep 17 14:17:03.926131 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.926115    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" event={"ID":"a201efd4-5330-4827-983e-da814079e228","Type":"ContainerDied","Data":"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"}
Sep 17 14:17:03.926294 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.926280    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f" event={"ID":"a201efd4-5330-4827-983e-da814079e228","Type":"ContainerDied","Data":"16586966d6c79e425b84ad73d2683cf43de87aec26f17d5763918d67059a934c"}
Sep 17 14:17:03.926486 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.926474    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"
Sep 17 14:17:03.943381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.943310    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtqfl\" (UniqueName: \"kubernetes.io/projected/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-kube-api-access-dtqfl\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:03.943381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.943334    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:03.943381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.943347    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1849fd6-7c97-4350-8fbe-f3ebae9ff412-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:03.943977 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.943958    2570 scope.go:117] "RemoveContainer" containerID="3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"
Sep 17 14:17:03.944586 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:17:03.944534    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224\": container with ID starting with 3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224 not found: ID does not exist" containerID="3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"
Sep 17 14:17:03.945117 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.945089    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224"} err="failed to get container status \"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224\": rpc error: code = NotFound desc = could not find container \"3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224\": container with ID starting with 3b2156be5136d89166087f56bbc38e035e6bf26ed03f8660b4c0d6aecdfde224 not found: ID does not exist"
Sep 17 14:17:03.945234 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.945223    2570 scope.go:117] "RemoveContainer" containerID="1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"
Sep 17 14:17:03.959910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.959884    2570 scope.go:117] "RemoveContainer" containerID="1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"
Sep 17 14:17:03.960257 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:17:03.960236    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d\": container with ID starting with 1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d not found: ID does not exist" containerID="1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"
Sep 17 14:17:03.960344 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.960263    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d"} err="failed to get container status \"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d\": rpc error: code = NotFound desc = could not find container \"1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d\": container with ID starting with 1c9663ab8b4a82ad89d663350bc24c03edf00e9be7b77a751c45cf8d63a4491d not found: ID does not exist"
Sep 17 14:17:03.967326 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.967296    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:17:03.980019 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:03.979993    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-767170-77cc4b8b9b-t9295"]
Sep 17 14:17:04.044051 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.044023    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume\") pod \"a201efd4-5330-4827-983e-da814079e228\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") "
Sep 17 14:17:04.044196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.044124    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7mpz\" (UniqueName: \"kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz\") pod \"a201efd4-5330-4827-983e-da814079e228\" (UID: \"a201efd4-5330-4827-983e-da814079e228\") "
Sep 17 14:17:04.044384 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.044364    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a201efd4-5330-4827-983e-da814079e228" (UID: "a201efd4-5330-4827-983e-da814079e228"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:17:04.046409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.046390    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz" (OuterVolumeSpecName: "kube-api-access-h7mpz") pod "a201efd4-5330-4827-983e-da814079e228" (UID: "a201efd4-5330-4827-983e-da814079e228"). InnerVolumeSpecName "kube-api-access-h7mpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:17:04.145058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.145006    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7mpz\" (UniqueName: \"kubernetes.io/projected/a201efd4-5330-4827-983e-da814079e228-kube-api-access-h7mpz\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:04.145058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.145052    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a201efd4-5330-4827-983e-da814079e228-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:17:04.182665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.182586    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c" path="/var/lib/kubelet/pods/0f6c6844-e2e9-4b60-9e8a-63d293c2ae5c/volumes"
Sep 17 14:17:04.183008 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.182994    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" path="/var/lib/kubelet/pods/e1849fd6-7c97-4350-8fbe-f3ebae9ff412/volumes"
Sep 17 14:17:04.256500 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.256468    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:17:04.268990 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:04.268954    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-767170-664795fc6b-pn62f"]
Sep 17 14:17:06.182250 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:06.182221    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a201efd4-5330-4827-983e-da814079e228" path="/var/lib/kubelet/pods/a201efd4-5330-4827-983e-da814079e228/volumes"
Sep 17 14:17:18.477351 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477318    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477708    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" containerName="maas-api"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477718    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" containerName="maas-api"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477744    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8008639b-4ee0-481c-9781-44458b721e06" containerName="payload-processing"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477751    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8008639b-4ee0-481c-9781-44458b721e06" containerName="payload-processing"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477761    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a201efd4-5330-4827-983e-da814079e228" containerName="payload-pre-processing"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477766    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a201efd4-5330-4827-983e-da814079e228" containerName="payload-pre-processing"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477832    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1849fd6-7c97-4350-8fbe-f3ebae9ff412" containerName="maas-api"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477839    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8008639b-4ee0-481c-9781-44458b721e06" containerName="payload-processing"
Sep 17 14:17:18.479963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.477847    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a201efd4-5330-4827-983e-da814079e228" containerName="payload-pre-processing"
Sep 17 14:17:18.481002 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.480980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.486149 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.486125    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-ebc24f-openshift-default-dockercfg-57665\""
Sep 17 14:17:18.498715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.498690    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:17:18.574446 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574415    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574446 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574458    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574476    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574535    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574605    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574627    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbtf4\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.574910 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.574763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675692 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675646    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675771    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675796    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.675926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676230 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676230 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.675983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbtf4\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676230 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.676182    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676230 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.676209    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676434 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.676351    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676434 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.676374    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.676617 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.676601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.678350 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.678331    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.678438 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.678377    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.698037 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.698010    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.700962 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.700935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbtf4\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4\") pod \"e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") " pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:18.793535 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:18.793460    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:19.076788 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:19.076760    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:17:19.078539 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:17:19.078509    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdb824c_e4f1_4e6a_beef_e4a6b725efa1.slice/crio-b51652b9bbd0b24c9e3ed4f4f49b694970217cd58cb7f345195a340bcc432c37 WatchSource:0}: Error finding container b51652b9bbd0b24c9e3ed4f4f49b694970217cd58cb7f345195a340bcc432c37: Status 404 returned error can't find the container with id b51652b9bbd0b24c9e3ed4f4f49b694970217cd58cb7f345195a340bcc432c37
Sep 17 14:17:19.081032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:19.081003    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:17:19.081107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:19.081065    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:17:19.081107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:19.081094    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:17:20.015664 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:20.015629    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" event={"ID":"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1","Type":"ContainerStarted","Data":"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"}
Sep 17 14:17:20.016083 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:20.015712    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" event={"ID":"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1","Type":"ContainerStarted","Data":"b51652b9bbd0b24c9e3ed4f4f49b694970217cd58cb7f345195a340bcc432c37"}
Sep 17 14:17:20.043370 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:20.043326    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" podStartSLOduration=2.043310881 podStartE2EDuration="2.043310881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:17:20.041896274 +0000 UTC m=+1282.278773749" watchObservedRunningTime="2026-09-17 14:17:20.043310881 +0000 UTC m=+1282.280188357"
Sep 17 14:17:20.794207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:20.794173    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:20.799344 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:20.799319    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:21.021276 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:21.021246    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:21.022345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:21.022324    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:17:26.819437 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:26.819398    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:17:27.635939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.635908    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:17:27.638363 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.638349    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.641095 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.641074    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2b8275\""
Sep 17 14:17:27.641201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.641074    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2b8275-dockercfg-b2j7q\""
Sep 17 14:17:27.650421 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.650400    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:17:27.764383 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.764350    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.764559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.764440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwkgt\" (UniqueName: \"kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.865824 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.865793    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.866263 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.865877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwkgt\" (UniqueName: \"kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.866484 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.866464    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.886838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.886779    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwkgt\" (UniqueName: \"kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt\") pod \"payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:27.949136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:27.949113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:28.228443 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:28.228422    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:17:28.230991 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:17:28.230965    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7059ca5d_dd99_454a_9216_86af0c7cbd96.slice/crio-02a1ec0bd338f8d257d31acc442b4951c46d23015f224b8772d2ab0da53cd2a8 WatchSource:0}: Error finding container 02a1ec0bd338f8d257d31acc442b4951c46d23015f224b8772d2ab0da53cd2a8: Status 404 returned error can't find the container with id 02a1ec0bd338f8d257d31acc442b4951c46d23015f224b8772d2ab0da53cd2a8
Sep 17 14:17:29.065336 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:29.065304    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" event={"ID":"7059ca5d-dd99-454a-9216-86af0c7cbd96","Type":"ContainerStarted","Data":"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"}
Sep 17 14:17:29.065756 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:29.065358    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" event={"ID":"7059ca5d-dd99-454a-9216-86af0c7cbd96","Type":"ContainerStarted","Data":"02a1ec0bd338f8d257d31acc442b4951c46d23015f224b8772d2ab0da53cd2a8"}
Sep 17 14:17:29.065756 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:29.065396    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:29.085332 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:29.085288    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" podStartSLOduration=2.085272085 podStartE2EDuration="2.085272085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:17:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:17:29.083597564 +0000 UTC m=+1291.320475042" watchObservedRunningTime="2026-09-17 14:17:29.085272085 +0000 UTC m=+1291.322149552"
Sep 17 14:17:40.071930 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:40.071893    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:17:52.024144 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:17:52.024108    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:18:03.100014 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.099985    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:18:03.104958 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.104939    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.107852 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.107834    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6e2bb9-dockercfg-8g5hs\""
Sep 17 14:18:03.108934 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.108916    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6e2bb9\""
Sep 17 14:18:03.120368 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.120348    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:18:03.227627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.227588    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d9mr\" (UniqueName: \"kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.227815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.227640    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.328545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.328512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d9mr\" (UniqueName: \"kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.328758 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.328571    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.329335 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.329316    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.338685 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.338661    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d9mr\" (UniqueName: \"kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr\") pod \"payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.415349 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.415317    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:03.729544 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:03.729507    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:18:03.732669 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:18:03.732640    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ff4d3c_4164_4965_975e_2bcb0457f594.slice/crio-88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d WatchSource:0}: Error finding container 88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d: Status 404 returned error can't find the container with id 88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d
Sep 17 14:18:04.259757 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:04.259682    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" event={"ID":"22ff4d3c-4164-4965-975e-2bcb0457f594","Type":"ContainerStarted","Data":"9afd5a34f7b4749afc48a8659e8c9eaaf2f74cda13ce8596f3f472c2ae56f32e"}
Sep 17 14:18:04.260161 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:04.259774    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" event={"ID":"22ff4d3c-4164-4965-975e-2bcb0457f594","Type":"ContainerStarted","Data":"88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d"}
Sep 17 14:18:04.260161 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:04.259803    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:04.282918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:04.282866    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" podStartSLOduration=1.282850559 podStartE2EDuration="1.282850559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:04.282048429 +0000 UTC m=+1326.518925916" watchObservedRunningTime="2026-09-17 14:18:04.282850559 +0000 UTC m=+1326.519728059"
Sep 17 14:18:15.267167 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:15.267140    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:18:21.015099 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:21.015068    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:18:35.039633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.039599    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7774-2d84ba"]
Sep 17 14:18:35.042913 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.042897    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:35.045928 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.045907    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f8z2f\""
Sep 17 14:18:35.050950 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.050925    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7774-2d84ba"]
Sep 17 14:18:35.147229 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.147195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s22t6\" (UniqueName: \"kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6\") pod \"test-curl-7774-2d84ba\" (UID: \"b6ea60fd-864f-4640-b425-18ff5497302f\") " pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:35.248266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.248236    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s22t6\" (UniqueName: \"kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6\") pod \"test-curl-7774-2d84ba\" (UID: \"b6ea60fd-864f-4640-b425-18ff5497302f\") " pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:35.259886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.259856    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s22t6\" (UniqueName: \"kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6\") pod \"test-curl-7774-2d84ba\" (UID: \"b6ea60fd-864f-4640-b425-18ff5497302f\") " pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:35.354872 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.354800    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:35.626434 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:35.626411    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7774-2d84ba"]
Sep 17 14:18:35.628907 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:18:35.628880    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6ea60fd_864f_4640_b425_18ff5497302f.slice/crio-4a277716a948ba544afc7155c3b60cf880f75dc107a0357c227054be3d141657 WatchSource:0}: Error finding container 4a277716a948ba544afc7155c3b60cf880f75dc107a0357c227054be3d141657: Status 404 returned error can't find the container with id 4a277716a948ba544afc7155c3b60cf880f75dc107a0357c227054be3d141657
Sep 17 14:18:36.093126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:36.093091    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:18:36.431896 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:36.431849    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7774-2d84ba" event={"ID":"b6ea60fd-864f-4640-b425-18ff5497302f","Type":"ContainerStarted","Data":"e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631"}
Sep 17 14:18:36.431896 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:36.431890    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7774-2d84ba" event={"ID":"b6ea60fd-864f-4640-b425-18ff5497302f","Type":"ContainerStarted","Data":"4a277716a948ba544afc7155c3b60cf880f75dc107a0357c227054be3d141657"}
Sep 17 14:18:36.456996 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:36.456947    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7774-2d84ba" podStartSLOduration=1.45693236 podStartE2EDuration="1.45693236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:36.456224253 +0000 UTC m=+1358.693101732" watchObservedRunningTime="2026-09-17 14:18:36.45693236 +0000 UTC m=+1358.693809850"
Sep 17 14:18:37.045286 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:37.045254    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7774-2d84ba"]
Sep 17 14:18:37.050287 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:37.050231    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7774-2d84ba"]
Sep 17 14:18:37.436896 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:37.436868    2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7774-2d84ba" container="test-curl-7774-2d84ba"
Sep 17 14:18:38.445460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:38.445420    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7774-2d84ba" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" containerName="test-curl-7774-2d84ba" containerID="cri-o://e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631" gracePeriod=2
Sep 17 14:18:40.629665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:40.629643    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:40.705928 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:40.705855    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s22t6\" (UniqueName: \"kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6\") pod \"b6ea60fd-864f-4640-b425-18ff5497302f\" (UID: \"b6ea60fd-864f-4640-b425-18ff5497302f\") "
Sep 17 14:18:40.708026 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:40.707998    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6" (OuterVolumeSpecName: "kube-api-access-s22t6") pod "b6ea60fd-864f-4640-b425-18ff5497302f" (UID: "b6ea60fd-864f-4640-b425-18ff5497302f"). InnerVolumeSpecName "kube-api-access-s22t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:18:40.806936 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:40.806909    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s22t6\" (UniqueName: \"kubernetes.io/projected/b6ea60fd-864f-4640-b425-18ff5497302f-kube-api-access-s22t6\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:41.463209 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:41.463168    2570 generic.go:356] "Generic (PLEG): container finished" podID="b6ea60fd-864f-4640-b425-18ff5497302f" containerID="e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631" exitCode=137
Sep 17 14:18:41.463209 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:41.463212    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:41.464232 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:41.463268    2570 scope.go:117] "RemoveContainer" containerID="e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631"
Sep 17 14:18:41.474023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:41.474004    2570 scope.go:117] "RemoveContainer" containerID="e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631"
Sep 17 14:18:41.474188 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:41.474158    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7774-2d84ba\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" pod="opendatahub/test-curl-7774-2d84ba"
Sep 17 14:18:41.474347 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:41.474317    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631\": container with ID starting with e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631 not found: ID does not exist" containerID="e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631"
Sep 17 14:18:41.474403 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:41.474347    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631"} err="failed to get container status \"e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631\": rpc error: code = NotFound desc = could not find container \"e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631\": container with ID starting with e37aad3aad701a27af076f1ea007b0ffc3644947859fae8d67c0efe2e6f8f631 not found: ID does not exist"
Sep 17 14:18:42.182352 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:42.182307    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" path="/var/lib/kubelet/pods/b6ea60fd-864f-4640-b425-18ff5497302f/volumes"
Sep 17 14:18:44.943662 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:44.943630    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:18:44.944154 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:44.943904    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" podUID="6574ab04-15cc-40aa-a3ec-788e6d692992" containerName="maas-api" containerID="cri-o://83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d" gracePeriod=30
Sep 17 14:18:45.005572 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.005542    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:18:45.005887 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.005840    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" podUID="591b56ff-da2c-4201-8142-7bb0c4f33201" containerName="payload-pre-processing" containerID="cri-o://7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42" gracePeriod=30
Sep 17 14:18:45.273550 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.273526    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:18:45.299554 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.299499    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:18:45.455387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455300    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28z6c\" (UniqueName: \"kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c\") pod \"591b56ff-da2c-4201-8142-7bb0c4f33201\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") "
Sep 17 14:18:45.455387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455368    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume\") pod \"591b56ff-da2c-4201-8142-7bb0c4f33201\" (UID: \"591b56ff-da2c-4201-8142-7bb0c4f33201\") "
Sep 17 14:18:45.455616 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455402    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrhb\" (UniqueName: \"kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb\") pod \"6574ab04-15cc-40aa-a3ec-788e6d692992\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") "
Sep 17 14:18:45.455616 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455436    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs\") pod \"6574ab04-15cc-40aa-a3ec-788e6d692992\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") "
Sep 17 14:18:45.455616 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455563    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca\") pod \"6574ab04-15cc-40aa-a3ec-788e6d692992\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") "
Sep 17 14:18:45.455814 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455668    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") pod \"6574ab04-15cc-40aa-a3ec-788e6d692992\" (UID: \"6574ab04-15cc-40aa-a3ec-788e6d692992\") "
Sep 17 14:18:45.455854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.455824    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "591b56ff-da2c-4201-8142-7bb0c4f33201" (UID: "591b56ff-da2c-4201-8142-7bb0c4f33201"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:18:45.456153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.456131    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/591b56ff-da2c-4201-8142-7bb0c4f33201-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:45.456266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.456147    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6574ab04-15cc-40aa-a3ec-788e6d692992" (UID: "6574ab04-15cc-40aa-a3ec-788e6d692992"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:18:45.458303 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.458268    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c" (OuterVolumeSpecName: "kube-api-access-28z6c") pod "591b56ff-da2c-4201-8142-7bb0c4f33201" (UID: "591b56ff-da2c-4201-8142-7bb0c4f33201"). InnerVolumeSpecName "kube-api-access-28z6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:18:45.458543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.458508    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6574ab04-15cc-40aa-a3ec-788e6d692992" (UID: "6574ab04-15cc-40aa-a3ec-788e6d692992"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:18:45.458831 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.458793    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb" (OuterVolumeSpecName: "kube-api-access-ncrhb") pod "6574ab04-15cc-40aa-a3ec-788e6d692992" (UID: "6574ab04-15cc-40aa-a3ec-788e6d692992"). InnerVolumeSpecName "kube-api-access-ncrhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:18:45.458924 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.458889    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6574ab04-15cc-40aa-a3ec-788e6d692992" (UID: "6574ab04-15cc-40aa-a3ec-788e6d692992"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:18:45.485358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.485330    2570 generic.go:356] "Generic (PLEG): container finished" podID="591b56ff-da2c-4201-8142-7bb0c4f33201" containerID="7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42" exitCode=0
Sep 17 14:18:45.485515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.485368    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"
Sep 17 14:18:45.485515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.485411    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" event={"ID":"591b56ff-da2c-4201-8142-7bb0c4f33201","Type":"ContainerDied","Data":"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"}
Sep 17 14:18:45.485515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.485472    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7" event={"ID":"591b56ff-da2c-4201-8142-7bb0c4f33201","Type":"ContainerDied","Data":"23123ef70ca80137dc70277d35a43b5269cf0851eaed6b1af4d9a2ebb73e5f97"}
Sep 17 14:18:45.485515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.485501    2570 scope.go:117] "RemoveContainer" containerID="7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"
Sep 17 14:18:45.486880 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.486862    2570 generic.go:356] "Generic (PLEG): container finished" podID="6574ab04-15cc-40aa-a3ec-788e6d692992" containerID="83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d" exitCode=0
Sep 17 14:18:45.486959 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.486908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" event={"ID":"6574ab04-15cc-40aa-a3ec-788e6d692992","Type":"ContainerDied","Data":"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"}
Sep 17 14:18:45.486959 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.486949    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw" event={"ID":"6574ab04-15cc-40aa-a3ec-788e6d692992","Type":"ContainerDied","Data":"ec91a593bee71c6676008adf93cfbce85e33234c97d95c67176384e770557ab6"}
Sep 17 14:18:45.487079 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.486916    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"
Sep 17 14:18:45.495472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.495456    2570 scope.go:117] "RemoveContainer" containerID="7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"
Sep 17 14:18:45.495766 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:45.495713    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42\": container with ID starting with 7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42 not found: ID does not exist" containerID="7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"
Sep 17 14:18:45.495859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.495812    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42"} err="failed to get container status \"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42\": rpc error: code = NotFound desc = could not find container \"7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42\": container with ID starting with 7deac579f982927730178b241ebb675420fea7c6ad74daba457993c5f30fca42 not found: ID does not exist"
Sep 17 14:18:45.495859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.495837    2570 scope.go:117] "RemoveContainer" containerID="83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"
Sep 17 14:18:45.504350 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.504335    2570 scope.go:117] "RemoveContainer" containerID="83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"
Sep 17 14:18:45.504606 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:45.504589    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d\": container with ID starting with 83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d not found: ID does not exist" containerID="83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"
Sep 17 14:18:45.504660 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.504613    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d"} err="failed to get container status \"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d\": rpc error: code = NotFound desc = could not find container \"83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d\": container with ID starting with 83ba8647075d550ebf7077b573e6379cc75ddfb00bbaafbe484b3641530e250d not found: ID does not exist"
Sep 17 14:18:45.510255 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.510236    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:18:45.514266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.514250    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-96cfc6-fccd494f8-skkd7"]
Sep 17 14:18:45.524595 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.524557    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:18:45.528422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.528406    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-96cfc6-68cb78f756-5cljw"]
Sep 17 14:18:45.557122 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.557096    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6574ab04-15cc-40aa-a3ec-788e6d692992-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:45.557122 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.557118    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:45.557260 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.557129    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28z6c\" (UniqueName: \"kubernetes.io/projected/591b56ff-da2c-4201-8142-7bb0c4f33201-kube-api-access-28z6c\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:45.557260 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.557137    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrhb\" (UniqueName: \"kubernetes.io/projected/6574ab04-15cc-40aa-a3ec-788e6d692992-kube-api-access-ncrhb\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:45.557260 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:45.557148    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6574ab04-15cc-40aa-a3ec-788e6d692992-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:46.182906 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:46.182875    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="591b56ff-da2c-4201-8142-7bb0c4f33201" path="/var/lib/kubelet/pods/591b56ff-da2c-4201-8142-7bb0c4f33201/volumes"
Sep 17 14:18:46.183297 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:46.183284    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6574ab04-15cc-40aa-a3ec-788e6d692992" path="/var/lib/kubelet/pods/6574ab04-15cc-40aa-a3ec-788e6d692992/volumes"
Sep 17 14:18:46.631416 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:46.631387    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:18:47.625411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625369    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:18:47.625914 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625890    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" containerName="test-curl-7774-2d84ba"
Sep 17 14:18:47.625914 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625913    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" containerName="test-curl-7774-2d84ba"
Sep 17 14:18:47.626057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625927    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6574ab04-15cc-40aa-a3ec-788e6d692992" containerName="maas-api"
Sep 17 14:18:47.626057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625936    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6574ab04-15cc-40aa-a3ec-788e6d692992" containerName="maas-api"
Sep 17 14:18:47.626057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625968    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="591b56ff-da2c-4201-8142-7bb0c4f33201" containerName="payload-pre-processing"
Sep 17 14:18:47.626057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.625977    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="591b56ff-da2c-4201-8142-7bb0c4f33201" containerName="payload-pre-processing"
Sep 17 14:18:47.626280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.626073    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="591b56ff-da2c-4201-8142-7bb0c4f33201" containerName="payload-pre-processing"
Sep 17 14:18:47.626280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.626088    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6ea60fd-864f-4640-b425-18ff5497302f" containerName="test-curl-7774-2d84ba"
Sep 17 14:18:47.626280 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.626096    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6574ab04-15cc-40aa-a3ec-788e6d692992" containerName="maas-api"
Sep 17 14:18:47.630834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.630814    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.633608 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.633586    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5cdd59e7-serving-cert\""
Sep 17 14:18:47.633753 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.633630    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-j4j8k\""
Sep 17 14:18:47.634641 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.634625    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 14:18:47.634725 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.634625    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 14:18:47.634878 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.634867    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 14:18:47.642086 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.642065    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:18:47.781078 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.781041    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.781078 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.781084    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn62v\" (UniqueName: \"kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.781302 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.781171    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.781302 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.781207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.882667 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.882587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.882667 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.882640    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.882913 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.882680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.882913 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.882701    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn62v\" (UniqueName: \"kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.882913 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:47.882765    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-serving-cert: secret "maas-api-e2e-mt-5cdd59e7-serving-cert" not found
Sep 17 14:18:47.882913 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:47.882845    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls podName:5a0be575-8145-4416-9750-dbf6712cb5bf nodeName:}" failed. No retries permitted until 2026-09-17 14:18:48.382823667 +0000 UTC m=+1370.619701141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls") pod "maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" (UID: "5a0be575-8145-4416-9750-dbf6712cb5bf") : secret "maas-api-e2e-mt-5cdd59e7-serving-cert" not found
Sep 17 14:18:47.883290 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.883272    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.885189 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.885173    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:47.891712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:47.891694    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn62v\" (UniqueName: \"kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:48.326047 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.326017    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:18:48.329715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.329700    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.332456 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.332434    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5cdd59e7-dockercfg-2xqs4\""
Sep 17 14:18:48.332593 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.332459    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5cdd59e7\""
Sep 17 14:18:48.342113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.342090    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:18:48.388421 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.388390    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:48.391016 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.390995    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") pod \"maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:48.489873 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.489839    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.490029 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.489930    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfvcn\" (UniqueName: \"kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.541354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.541317    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:48.591859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.591196    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfvcn\" (UniqueName: \"kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.591859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.591307    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.592284 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.592256    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.601633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.601604    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfvcn\" (UniqueName: \"kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn\") pod \"payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") " pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.641137 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.641111    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:48.837830 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.837802    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:18:48.839884 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:18:48.839818    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0be575_8145_4416_9750_dbf6712cb5bf.slice/crio-a106056a647f9e605164f0fa2aee5a42c8f820a82c98e4ca05437950443a9fa5 WatchSource:0}: Error finding container a106056a647f9e605164f0fa2aee5a42c8f820a82c98e4ca05437950443a9fa5: Status 404 returned error can't find the container with id a106056a647f9e605164f0fa2aee5a42c8f820a82c98e4ca05437950443a9fa5
Sep 17 14:18:48.841344 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.841327    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:18:48.923117 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:48.923098    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:18:48.924994 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:18:48.924966    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec96a75e_53f5_41f8_a515_86320f577ba0.slice/crio-b041570493666ca0c640ff3a1dc53d9d777cbaaa355cbeba03f92ba4b7113aa6 WatchSource:0}: Error finding container b041570493666ca0c640ff3a1dc53d9d777cbaaa355cbeba03f92ba4b7113aa6: Status 404 returned error can't find the container with id b041570493666ca0c640ff3a1dc53d9d777cbaaa355cbeba03f92ba4b7113aa6
Sep 17 14:18:49.514239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.514206    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" event={"ID":"ec96a75e-53f5-41f8-a515-86320f577ba0","Type":"ContainerStarted","Data":"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"}
Sep 17 14:18:49.514409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.514348    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:18:49.514503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.514407    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" event={"ID":"ec96a75e-53f5-41f8-a515-86320f577ba0","Type":"ContainerStarted","Data":"b041570493666ca0c640ff3a1dc53d9d777cbaaa355cbeba03f92ba4b7113aa6"}
Sep 17 14:18:49.515859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.515836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" event={"ID":"5a0be575-8145-4416-9750-dbf6712cb5bf","Type":"ContainerStarted","Data":"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"}
Sep 17 14:18:49.515965 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.515880    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" event={"ID":"5a0be575-8145-4416-9750-dbf6712cb5bf","Type":"ContainerStarted","Data":"a106056a647f9e605164f0fa2aee5a42c8f820a82c98e4ca05437950443a9fa5"}
Sep 17 14:18:49.515965 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.515905    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:18:49.535176 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.535133    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" podStartSLOduration=1.535118676 podStartE2EDuration="1.535118676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:49.532645526 +0000 UTC m=+1371.769523005" watchObservedRunningTime="2026-09-17 14:18:49.535118676 +0000 UTC m=+1371.771996178"
Sep 17 14:18:49.557153 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:49.557095    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" podStartSLOduration=2.019811673 podStartE2EDuration="2.557075453s" totalImagesPullingTime="537.26378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:18:49.553767391 +0000 UTC m=+1371.790644866" watchObservedRunningTime="2026-09-17 14:18:49.557075453 +0000 UTC m=+1371.793952931"
Sep 17 14:18:51.112606 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.112549    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" podUID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" containerName="main" containerID="cri-o://f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118" gracePeriod=45
Sep 17 14:18:51.367249 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.367189    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:18:51.417293 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417266    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417448 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417316    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417448 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417339    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417448 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417368    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417448 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417426    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417482    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kd9r\" (UniqueName: \"kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r\") pod \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\" (UID: \"78bb0f84-029b-48eb-92fa-6a638eeb63bb\") "
Sep 17 14:18:51.417656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417575    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:18:51.417656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417584    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home" (OuterVolumeSpecName: "home") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:18:51.417870 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417816    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:18:51.417929 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417900    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.417929 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.417921    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.419953 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.419930    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r" (OuterVolumeSpecName: "kube-api-access-6kd9r") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "kube-api-access-6kd9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:18:51.420061 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.420000    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:18:51.420106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.420056    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm" (OuterVolumeSpecName: "dshm") pod "78bb0f84-029b-48eb-92fa-6a638eeb63bb" (UID: "78bb0f84-029b-48eb-92fa-6a638eeb63bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:18:51.519185 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.519149    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.519185 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.519178    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kd9r\" (UniqueName: \"kubernetes.io/projected/78bb0f84-029b-48eb-92fa-6a638eeb63bb-kube-api-access-6kd9r\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.519185 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.519192    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78bb0f84-029b-48eb-92fa-6a638eeb63bb-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.519417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.519204    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78bb0f84-029b-48eb-92fa-6a638eeb63bb-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:18:51.527911 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.527885    2570 generic.go:356] "Generic (PLEG): container finished" podID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" containerID="f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118" exitCode=0
Sep 17 14:18:51.528076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.527921    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" event={"ID":"78bb0f84-029b-48eb-92fa-6a638eeb63bb","Type":"ContainerDied","Data":"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"}
Sep 17 14:18:51.528076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.527929    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"
Sep 17 14:18:51.528076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.527952    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr" event={"ID":"78bb0f84-029b-48eb-92fa-6a638eeb63bb","Type":"ContainerDied","Data":"9e845aff21d9d05020d44621e508e1eb4a902119254560b9521218b7984275df"}
Sep 17 14:18:51.528076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.527975    2570 scope.go:117] "RemoveContainer" containerID="f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"
Sep 17 14:18:51.538240 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.538217    2570 scope.go:117] "RemoveContainer" containerID="f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"
Sep 17 14:18:51.538512 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:51.538493    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118\": container with ID starting with f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118 not found: ID does not exist" containerID="f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"
Sep 17 14:18:51.538576 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.538525    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118"} err="failed to get container status \"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118\": rpc error: code = NotFound desc = could not find container \"f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118\": container with ID starting with f603bdd0722d0f9c1b2971b499df6cc9d490ef75166a7a70c725dac07503c118 not found: ID does not exist"
Sep 17 14:18:51.565070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.565012    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:18:51.567797 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:51.567775    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-96cfc6/test-model-96cfc6-kserve-79d4749499-ntkrr"]
Sep 17 14:18:51.576187 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:18:51.576167    2570 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78bb0f84_029b_48eb_92fa_6a638eeb63bb.slice\": RecentStats: unable to find data in memory cache]"
Sep 17 14:18:52.182325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:52.182297    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" path="/var/lib/kubelet/pods/78bb0f84-029b-48eb-92fa-6a638eeb63bb/volumes"
Sep 17 14:18:55.526501 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:18:55.526475    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:19:00.523613 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:00.523586    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:19:08.887007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:08.886976    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:19:08.887534 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:08.887171    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" podUID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" containerName="istio-proxy" containerID="cri-o://9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948" gracePeriod=30
Sep 17 14:19:11.816237 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:11.816202    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"]
Sep 17 14:19:11.820044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:11.820014    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-fe6ee5-29827575-fvrr4"]
Sep 17 14:19:11.856336 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:11.856305    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:19:11.856622 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:11.856591    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" podUID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" containerName="payload-pre-processing" containerID="cri-o://6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97" gracePeriod=30
Sep 17 14:19:12.114109 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.114085    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:19:12.185330 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.184943    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d536a4b9-9fa4-46fb-907b-5106890d0653" path="/var/lib/kubelet/pods/d536a4b9-9fa4-46fb-907b-5106890d0653/volumes"
Sep 17 14:19:12.232503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.232457    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lg4v\" (UniqueName: \"kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v\") pod \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") "
Sep 17 14:19:12.232707 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.232630    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume\") pod \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\" (UID: \"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed\") "
Sep 17 14:19:12.233368 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.233321    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" (UID: "fd9198de-a972-4e6c-9d3c-0f41fb7a40ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:19:12.235963 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.235935    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v" (OuterVolumeSpecName: "kube-api-access-2lg4v") pod "fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" (UID: "fd9198de-a972-4e6c-9d3c-0f41fb7a40ed"). InnerVolumeSpecName "kube-api-access-2lg4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:12.333883 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.333853    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lg4v\" (UniqueName: \"kubernetes.io/projected/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-kube-api-access-2lg4v\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:12.333883 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.333879    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:12.639424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.639394    2570 generic.go:356] "Generic (PLEG): container finished" podID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" containerID="6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97" exitCode=0
Sep 17 14:19:12.639571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.639440    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"
Sep 17 14:19:12.639571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.639481    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" event={"ID":"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed","Type":"ContainerDied","Data":"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"}
Sep 17 14:19:12.639571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.639538    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg" event={"ID":"fd9198de-a972-4e6c-9d3c-0f41fb7a40ed","Type":"ContainerDied","Data":"8810cebf76004ba1994508e8c47b13007226557f9ea015b88bd532ca6edbcd27"}
Sep 17 14:19:12.639571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.639567    2570 scope.go:117] "RemoveContainer" containerID="6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"
Sep 17 14:19:12.648479 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.648459    2570 scope.go:117] "RemoveContainer" containerID="6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"
Sep 17 14:19:12.648765 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:19:12.648726    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97\": container with ID starting with 6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97 not found: ID does not exist" containerID="6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"
Sep 17 14:19:12.648823 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.648776    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97"} err="failed to get container status \"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97\": rpc error: code = NotFound desc = could not find container \"6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97\": container with ID starting with 6e6ded574b5755029772c688abc171201034a459afd6adada610dda5bbdb5a97 not found: ID does not exist"
Sep 17 14:19:12.665552 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.665516    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:19:12.667661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:12.667641    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-fe6ee5-5656d899b6-hdnrg"]
Sep 17 14:19:14.154626 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.154601    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:19:14.181950 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.181924    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" path="/var/lib/kubelet/pods/fd9198de-a972-4e6c-9d3c-0f41fb7a40ed/volumes"
Sep 17 14:19:14.253007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.252972    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253007 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253009    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbtf4\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253042    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253093    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253142    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253180    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253251 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253213    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253508 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253254    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253508 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253281    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo\") pod \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\" (UID: \"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1\") "
Sep 17 14:19:14.253508 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253422    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:19:14.253668 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253528    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:14.253775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253727    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:14.253775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253763    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:14.254126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.253993    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.254126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.254014    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.254126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.254027    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.254126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.254041    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.254126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.254095    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:14.256253 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.256220    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:19:14.256253 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.256225    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:14.256417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.256285    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:14.256417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.256297    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4" (OuterVolumeSpecName: "kube-api-access-bbtf4") pod "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" (UID: "bbdb824c-e4f1-4e6a-beef-e4a6b725efa1"). InnerVolumeSpecName "kube-api-access-bbtf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:14.354974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.354933    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbtf4\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-kube-api-access-bbtf4\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.354974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.354966    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.354974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.354979    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.355338 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.354992    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.355338 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.355008    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:14.652646 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.652609    2570 generic.go:356] "Generic (PLEG): container finished" podID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" containerID="9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948" exitCode=0
Sep 17 14:19:14.652840 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.652662    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"
Sep 17 14:19:14.652840 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.652690    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" event={"ID":"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1","Type":"ContainerDied","Data":"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"}
Sep 17 14:19:14.652840 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.652777    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf" event={"ID":"bbdb824c-e4f1-4e6a-beef-e4a6b725efa1","Type":"ContainerDied","Data":"b51652b9bbd0b24c9e3ed4f4f49b694970217cd58cb7f345195a340bcc432c37"}
Sep 17 14:19:14.652840 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.652807    2570 scope.go:117] "RemoveContainer" containerID="9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"
Sep 17 14:19:14.663208 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.663188    2570 scope.go:117] "RemoveContainer" containerID="9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"
Sep 17 14:19:14.663484 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:19:14.663464    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948\": container with ID starting with 9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948 not found: ID does not exist" containerID="9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"
Sep 17 14:19:14.663554 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.663495    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948"} err="failed to get container status \"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948\": rpc error: code = NotFound desc = could not find container \"9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948\": container with ID starting with 9766630d6929dab03b416db36925b9bd7669e2f8ec782854e961e6654e170948 not found: ID does not exist"
Sep 17 14:19:14.678459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.678432    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:19:14.686032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:14.686008    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-ebc24f-openshift-default-dc6ff6d9b-dwxwf"]
Sep 17 14:19:16.182052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:16.182021    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" path="/var/lib/kubelet/pods/bbdb824c-e4f1-4e6a-beef-e4a6b725efa1/volumes"
Sep 17 14:19:19.560128 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:19.560098    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:19:19.560559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:19.560325    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" containerID="cri-o://46abae3a365952795ba45aa15f4104ae22df29dd5046d6b645ae8b46fc842986" gracePeriod=30
Sep 17 14:19:20.433008 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.432977    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:19:20.433364 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433348    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" containerName="payload-pre-processing"
Sep 17 14:19:20.433463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433366    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" containerName="payload-pre-processing"
Sep 17 14:19:20.433463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433391    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" containerName="main"
Sep 17 14:19:20.433463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433397    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" containerName="main"
Sep 17 14:19:20.433463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433411    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" containerName="istio-proxy"
Sep 17 14:19:20.433463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433417    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" containerName="istio-proxy"
Sep 17 14:19:20.433747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433473    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="78bb0f84-029b-48eb-92fa-6a638eeb63bb" containerName="main"
Sep 17 14:19:20.433747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433482    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbdb824c-e4f1-4e6a-beef-e4a6b725efa1" containerName="istio-proxy"
Sep 17 14:19:20.433747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.433489    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd9198de-a972-4e6c-9d3c-0f41fb7a40ed" containerName="payload-pre-processing"
Sep 17 14:19:20.437820 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.437796    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.440628 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.440610    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b88b90-openshift-default-dockercfg-j8wjp\""
Sep 17 14:19:20.449949 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.449928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:19:20.615089 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615054    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqq6k\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615158    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615188    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615219    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615285    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615333    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.615504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.615387    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716177 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716177 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716157    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716188    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqq6k\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716210    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716226    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716618 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716554    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716855 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716672    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716883    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.716978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.717211 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.716966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.718782 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.718760    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.718971 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.718952    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.725156 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.725127    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.725838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.725821    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqq6k\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k\") pod \"e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") " pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:20.749776 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:20.749748    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:21.034323 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.034294    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:19:21.036032 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:19:21.036007    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f20f97_1e63_4821_9d54_874aab9eeaec.slice/crio-3d55ee0bba58f8da5abe5a154abf88454ebed8717931318732315c9aca31becc WatchSource:0}: Error finding container 3d55ee0bba58f8da5abe5a154abf88454ebed8717931318732315c9aca31becc: Status 404 returned error can't find the container with id 3d55ee0bba58f8da5abe5a154abf88454ebed8717931318732315c9aca31becc
Sep 17 14:19:21.038356 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.038326    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:21.038431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.038391    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:21.038469 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.038438    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:21.692706 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.692669    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerStarted","Data":"0d5777be4a39449caba0a8a0e7b3626f04fa24ecfa620aab2cb7caa57521407d"}
Sep 17 14:19:21.693155 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.692775    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerStarted","Data":"3d55ee0bba58f8da5abe5a154abf88454ebed8717931318732315c9aca31becc"}
Sep 17 14:19:21.723284 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.723237    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podStartSLOduration=1.723222021 podStartE2EDuration="1.723222021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:21.720666644 +0000 UTC m=+1403.957544123" watchObservedRunningTime="2026-09-17 14:19:21.723222021 +0000 UTC m=+1403.960099498"
Sep 17 14:19:21.749984 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:21.749950    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:22.756960 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:22.756931    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:23.216331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:23.216292    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:19:23.704367 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:23.704336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:23.705303 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:23.705283    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:24.329145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.329109    2570 patch_prober.go:28] interesting pod/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 17 14:19:24.329503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.329160    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 14:19:24.712966 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.712930    2570 generic.go:356] "Generic (PLEG): container finished" podID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerID="46abae3a365952795ba45aa15f4104ae22df29dd5046d6b645ae8b46fc842986" exitCode=0
Sep 17 14:19:24.713129 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.712999    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerDied","Data":"46abae3a365952795ba45aa15f4104ae22df29dd5046d6b645ae8b46fc842986"}
Sep 17 14:19:24.713129 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.713053    2570 scope.go:117] "RemoveContainer" containerID="6230bc1b451f4017b2e00ce01c33e4cd6c104e2ea5bf6eaa38c192b53b8b1098"
Sep 17 14:19:24.818263 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.818240    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:19:24.856110 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856016    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856135    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nk9q\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856181    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856212    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856239    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856273    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856298    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data" (OuterVolumeSpecName: "istio-data") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:24.856476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856320    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856356    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856409    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket\") pod \"8adbfc9f-822d-484a-8f53-60ae83e0e394\" (UID: \"8adbfc9f-822d-484a-8f53-60ae83e0e394\") "
Sep 17 14:19:24.856720 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856491    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:24.856837 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856810    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:19:24.856926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856870    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.856926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.856889    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.857145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.857111    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:24.857292 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.857273    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:24.859417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.859393    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:19:24.859511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.859428    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token" (OuterVolumeSpecName: "istio-token") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:24.859653 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.859634    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:19:24.859653 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.859646    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q" (OuterVolumeSpecName: "kube-api-access-2nk9q") pod "8adbfc9f-822d-484a-8f53-60ae83e0e394" (UID: "8adbfc9f-822d-484a-8f53-60ae83e0e394"). InnerVolumeSpecName "kube-api-access-2nk9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:19:24.957461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957423    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957458    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8adbfc9f-822d-484a-8f53-60ae83e0e394-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957467    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957693 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957476    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957693 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957486    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8adbfc9f-822d-484a-8f53-60ae83e0e394-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957693 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957494    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nk9q\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-kube-api-access-2nk9q\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:24.957693 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:24.957503    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8adbfc9f-822d-484a-8f53-60ae83e0e394-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:19:25.719036 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:25.719006    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"
Sep 17 14:19:25.719523 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:25.719049    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp" event={"ID":"8adbfc9f-822d-484a-8f53-60ae83e0e394","Type":"ContainerDied","Data":"362de29b96fac96ff9c3208df0f57294a9f1f3257877bc335eb5b5af28f11f9c"}
Sep 17 14:19:25.719523 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:25.719119    2570 scope.go:117] "RemoveContainer" containerID="46abae3a365952795ba45aa15f4104ae22df29dd5046d6b645ae8b46fc842986"
Sep 17 14:19:25.742756 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:25.742715    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:19:25.746493 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:25.746471    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-fe6ee5-openshift-default-789585b85b-t96mp"]
Sep 17 14:19:26.182419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:26.182383    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" path="/var/lib/kubelet/pods/8adbfc9f-822d-484a-8f53-60ae83e0e394/volumes"
Sep 17 14:19:29.616214 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.616185    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:19:29.616601 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.616589    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:29.616647 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.616603    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:29.616689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.616680    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:29.619632 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.619615    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.622825 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.622809    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-b88b90-dockercfg-fbkqs\""
Sep 17 14:19:29.623226 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.623211    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-b88b90\""
Sep 17 14:19:29.632390 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.632362    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:19:29.703086 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.703056    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpsn4\" (UniqueName: \"kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.703478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.703455    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.804656 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.804624    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.804859 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.804685    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpsn4\" (UniqueName: \"kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.805361 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.805341    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.814113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.814092    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpsn4\" (UniqueName: \"kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4\") pod \"payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:29.930704 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:29.930676    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:30.228597 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:30.228572    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:19:30.230943 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:19:30.230911    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bb8d85_0ece_43e4_8ac5_c08ef07a9170.slice/crio-d5bbbb1623c43c1bc7daeb3f0756e383c0336db537987ca6dc52bb64caa26026 WatchSource:0}: Error finding container d5bbbb1623c43c1bc7daeb3f0756e383c0336db537987ca6dc52bb64caa26026: Status 404 returned error can't find the container with id d5bbbb1623c43c1bc7daeb3f0756e383c0336db537987ca6dc52bb64caa26026
Sep 17 14:19:30.746868 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:30.746830    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" event={"ID":"34bb8d85-0ece-43e4-8ac5-c08ef07a9170","Type":"ContainerStarted","Data":"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"}
Sep 17 14:19:30.747240 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:30.746886    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" event={"ID":"34bb8d85-0ece-43e4-8ac5-c08ef07a9170","Type":"ContainerStarted","Data":"d5bbbb1623c43c1bc7daeb3f0756e383c0336db537987ca6dc52bb64caa26026"}
Sep 17 14:19:30.747240 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:30.746935    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:30.768523 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:30.768474    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" podStartSLOduration=1.768458509 podStartE2EDuration="1.768458509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:19:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:19:30.76566816 +0000 UTC m=+1413.002545638" watchObservedRunningTime="2026-09-17 14:19:30.768458509 +0000 UTC m=+1413.005335987"
Sep 17 14:19:41.756002 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:41.755974    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:19:45.031354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:45.031316    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:19:54.745921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:54.745877    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 17 14:19:54.746421 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:54.745965    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 17 14:19:56.899297 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.899205    2570 generic.go:356] "Generic (PLEG): container finished" podID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerID="0d5777be4a39449caba0a8a0e7b3626f04fa24ecfa620aab2cb7caa57521407d" exitCode=0
Sep 17 14:19:56.899297 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.899276    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerDied","Data":"0d5777be4a39449caba0a8a0e7b3626f04fa24ecfa620aab2cb7caa57521407d"}
Sep 17 14:19:56.899727 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.899699    2570 scope.go:117] "RemoveContainer" containerID="0d5777be4a39449caba0a8a0e7b3626f04fa24ecfa620aab2cb7caa57521407d"
Sep 17 14:19:56.900894 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.900870    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:56.900956 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.900932    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:56.900993 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:56.900961    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:19:57.906546 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:57.906510    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerStarted","Data":"8a83b04362fcc105f9b8c60f2206cac75d83dce3356f874b2211332f311bf072"}
Sep 17 14:19:58.750570 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:58.750539    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:19:58.750984 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:58.750885    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 17 14:19:58.751196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:58.751170    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 17 14:19:59.504770 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.504553    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:19:59.505139 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.504999    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:59.505139 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.505013    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:59.505139 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.505087    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8adbfc9f-822d-484a-8f53-60ae83e0e394" containerName="istio-proxy"
Sep 17 14:19:59.507858 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.507837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.510807 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.510788    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-b79988-serving-cert\""
Sep 17 14:19:59.525108 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.525081    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:19:59.601531 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.601503    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.601713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.601537    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.601713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.601565    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.601713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.601638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qx9n\" (UniqueName: \"kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.702203 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.702173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.702203 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.702209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.702416 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.702240    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.702416 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.702285    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qx9n\" (UniqueName: \"kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.702416 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:19:59.702372    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-serving-cert: secret "maas-api-e2e-shared-b-w3-b79988-serving-cert" not found
Sep 17 14:19:59.702540 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:19:59.702451    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls podName:5d310bf0-958d-4797-93b1-434b7ff189a0 nodeName:}" failed. No retries permitted until 2026-09-17 14:20:00.20243009 +0000 UTC m=+1442.439307545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls") pod "maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" (UID: "5d310bf0-958d-4797-93b1-434b7ff189a0") : secret "maas-api-e2e-shared-b-w3-b79988-serving-cert" not found
Sep 17 14:19:59.702885 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.702862    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.704893 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.704871    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.715994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.715971    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qx9n\" (UniqueName: \"kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:19:59.750613 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.750577    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 17 14:19:59.750766 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:19:59.750639    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 17 14:20:00.206589 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.206555    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:20:00.209025 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.208997    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:20:00.389508 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.389478    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:20:00.392259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.392244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.394903 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.394879    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-b79988\""
Sep 17 14:20:00.395035 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.394880    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-b79988-dockercfg-tcdh7\""
Sep 17 14:20:00.403787 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.403767    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:20:00.408540 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.408507    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.408747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.408708    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8p67\" (UniqueName: \"kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.421062 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.421043    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:20:00.510381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.509391    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p67\" (UniqueName: \"kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.510381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.509454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.510993 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.510957    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.521102 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.521077    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8p67\" (UniqueName: \"kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67\") pod \"payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.703461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.703428    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:00.742455 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.742383    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:20:00.744469 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:20:00.744433    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d310bf0_958d_4797_93b1_434b7ff189a0.slice/crio-48f77743af2f23446f714f1f5c455d179eac40bc25319148b318fa216d35478f WatchSource:0}: Error finding container 48f77743af2f23446f714f1f5c455d179eac40bc25319148b318fa216d35478f: Status 404 returned error can't find the container with id 48f77743af2f23446f714f1f5c455d179eac40bc25319148b318fa216d35478f
Sep 17 14:20:00.754207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.754059    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 17 14:20:00.754207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.754120    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 17 14:20:00.923751 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:00.923700    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" event={"ID":"5d310bf0-958d-4797-93b1-434b7ff189a0","Type":"ContainerStarted","Data":"48f77743af2f23446f714f1f5c455d179eac40bc25319148b318fa216d35478f"}
Sep 17 14:20:01.000352 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.000323    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:20:01.003564 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:20:01.003533    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531ead4a_772b_43f4_8642_26530715cddf.slice/crio-4495dd5e53cf2087db6742fafafb95ff8a7d515912d879bb75617efd670f8f54 WatchSource:0}: Error finding container 4495dd5e53cf2087db6742fafafb95ff8a7d515912d879bb75617efd670f8f54: Status 404 returned error can't find the container with id 4495dd5e53cf2087db6742fafafb95ff8a7d515912d879bb75617efd670f8f54
Sep 17 14:20:01.754404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.754357    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 17 14:20:01.754793 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.754433    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 17 14:20:01.931048 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.931014    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" event={"ID":"5d310bf0-958d-4797-93b1-434b7ff189a0","Type":"ContainerStarted","Data":"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"}
Sep 17 14:20:01.931258 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.931076    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:20:01.932710 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.932675    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" event={"ID":"531ead4a-772b-43f4-8642-26530715cddf","Type":"ContainerStarted","Data":"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"}
Sep 17 14:20:01.932854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.932716    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" event={"ID":"531ead4a-772b-43f4-8642-26530715cddf","Type":"ContainerStarted","Data":"4495dd5e53cf2087db6742fafafb95ff8a7d515912d879bb75617efd670f8f54"}
Sep 17 14:20:01.932854 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.932782    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:01.983527 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.983410    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" podStartSLOduration=1.98338967 podStartE2EDuration="1.98338967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:20:01.9822694 +0000 UTC m=+1444.219146878" watchObservedRunningTime="2026-09-17 14:20:01.98338967 +0000 UTC m=+1444.220267150"
Sep 17 14:20:01.984567 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:01.984519    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" podStartSLOduration=2.038214391 podStartE2EDuration="2.984501146s" totalImagesPullingTime="946.286755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:20:01.958963057 +0000 UTC m=+1444.195840533" watchObservedRunningTime="2026-09-17 14:20:01.984501146 +0000 UTC m=+1444.221378624"
Sep 17 14:20:02.751203 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:02.751167    2570 patch_prober.go:28] interesting pod/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 17 14:20:02.751487 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:02.751459    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 17 14:20:03.755396 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:03.755364    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:20:03.755861 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:03.755657    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:20:03.756413 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:03.756397    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:20:07.942571 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:07.942541    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:20:12.939692 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:12.939666    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:20:31.427187 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:20:31.427156    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:21:00.410974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.410942    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:21:00.414759 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.414711    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.417562 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.417540    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"kube-root-ca.crt\""
Sep 17 14:21:00.419023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.419000    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"openshift-service-ca.crt\""
Sep 17 14:21:00.419189 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.419025    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"auth-test-model-b88b90-kserve-self-signed-certs\""
Sep 17 14:21:00.419309 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.419044    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"default-dockercfg-l84qx\""
Sep 17 14:21:00.425683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.425663    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:21:00.493831 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.493796    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.493995 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.493851    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.493995 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.493963    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.494070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.494002    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gj8x\" (UniqueName: \"kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.494070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.494031    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.494142 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.494106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595350 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gj8x\" (UniqueName: \"kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595356    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595495    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595558    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595906 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595967 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595923    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.595967 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.595943    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.597899 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.597877    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.598150 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.598135    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.604405 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.604387    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gj8x\" (UniqueName: \"kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x\") pod \"auth-test-model-b88b90-kserve-5784ddb667-szjvj\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:00.727097 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:00.727010    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:01.013349 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:01.013325    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:21:01.015536 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:21:01.015493    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4246008e_ef59_49c7_b98e_382c40f70986.slice/crio-f4a1379aab8a510cede34764363deec30cd67def9942679d2e3e3c929f240191 WatchSource:0}: Error finding container f4a1379aab8a510cede34764363deec30cd67def9942679d2e3e3c929f240191: Status 404 returned error can't find the container with id f4a1379aab8a510cede34764363deec30cd67def9942679d2e3e3c929f240191
Sep 17 14:21:01.260662 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:01.260630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" event={"ID":"4246008e-ef59-49c7-b98e-382c40f70986","Type":"ContainerStarted","Data":"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"}
Sep 17 14:21:01.260882 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:01.260674    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" event={"ID":"4246008e-ef59-49c7-b98e-382c40f70986","Type":"ContainerStarted","Data":"f4a1379aab8a510cede34764363deec30cd67def9942679d2e3e3c929f240191"}
Sep 17 14:21:01.279926 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:01.279820    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" podStartSLOduration=1.279802979 podStartE2EDuration="1.279802979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:21:01.278811064 +0000 UTC m=+1503.515688542" watchObservedRunningTime="2026-09-17 14:21:01.279802979 +0000 UTC m=+1503.516680465"
Sep 17 14:21:07.609437 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:07.609400    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:21:10.727388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:10.727336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:10.727900 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:10.727405    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:10.740296 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:10.740271    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:11.326523 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:11.326495    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:21:28.535722 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.535687    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:21:28.536218 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.535978    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" podUID="5a0be575-8145-4416-9750-dbf6712cb5bf" containerName="maas-api" containerID="cri-o://5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f" gracePeriod=30
Sep 17 14:21:28.670758 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.670703    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:21:28.670983 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.670957    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" podUID="ec96a75e-53f5-41f8-a515-86320f577ba0" containerName="payload-processing" containerID="cri-o://f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec" gracePeriod=30
Sep 17 14:21:28.817384 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.817358    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:21:28.878420 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.878387    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn62v\" (UniqueName: \"kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v\") pod \"5a0be575-8145-4416-9750-dbf6712cb5bf\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") "
Sep 17 14:21:28.878607 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.878510    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca\") pod \"5a0be575-8145-4416-9750-dbf6712cb5bf\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") "
Sep 17 14:21:28.878607 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.878547    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs\") pod \"5a0be575-8145-4416-9750-dbf6712cb5bf\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") "
Sep 17 14:21:28.878607 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.878597    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") pod \"5a0be575-8145-4416-9750-dbf6712cb5bf\" (UID: \"5a0be575-8145-4416-9750-dbf6712cb5bf\") "
Sep 17 14:21:28.878939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.878900    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a0be575-8145-4416-9750-dbf6712cb5bf" (UID: "5a0be575-8145-4416-9750-dbf6712cb5bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:21:28.881312 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.881282    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5a0be575-8145-4416-9750-dbf6712cb5bf" (UID: "5a0be575-8145-4416-9750-dbf6712cb5bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:21:28.882466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.882431    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a0be575-8145-4416-9750-dbf6712cb5bf" (UID: "5a0be575-8145-4416-9750-dbf6712cb5bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:21:28.882466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.882437    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v" (OuterVolumeSpecName: "kube-api-access-hn62v") pod "5a0be575-8145-4416-9750-dbf6712cb5bf" (UID: "5a0be575-8145-4416-9750-dbf6712cb5bf"). InnerVolumeSpecName "kube-api-access-hn62v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:21:28.911770 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.911746    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:21:28.979850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.979820    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume\") pod \"ec96a75e-53f5-41f8-a515-86320f577ba0\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") "
Sep 17 14:21:28.980054 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.979896    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfvcn\" (UniqueName: \"kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn\") pod \"ec96a75e-53f5-41f8-a515-86320f577ba0\" (UID: \"ec96a75e-53f5-41f8-a515-86320f577ba0\") "
Sep 17 14:21:28.980249 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.980232    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:28.980316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.980255    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn62v\" (UniqueName: \"kubernetes.io/projected/5a0be575-8145-4416-9750-dbf6712cb5bf-kube-api-access-hn62v\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:28.980316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.980268    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a0be575-8145-4416-9750-dbf6712cb5bf-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:28.980316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.980264    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec96a75e-53f5-41f8-a515-86320f577ba0" (UID: "ec96a75e-53f5-41f8-a515-86320f577ba0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:21:28.980316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.980281    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5a0be575-8145-4416-9750-dbf6712cb5bf-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:28.982126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:28.982103    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn" (OuterVolumeSpecName: "kube-api-access-dfvcn") pod "ec96a75e-53f5-41f8-a515-86320f577ba0" (UID: "ec96a75e-53f5-41f8-a515-86320f577ba0"). InnerVolumeSpecName "kube-api-access-dfvcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:21:29.081098 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.081009    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec96a75e-53f5-41f8-a515-86320f577ba0-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:29.081098 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.081040    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfvcn\" (UniqueName: \"kubernetes.io/projected/ec96a75e-53f5-41f8-a515-86320f577ba0-kube-api-access-dfvcn\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:21:29.416506 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.416469    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec96a75e-53f5-41f8-a515-86320f577ba0" containerID="f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec" exitCode=0
Sep 17 14:21:29.416716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.416539    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"
Sep 17 14:21:29.416716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.416561    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" event={"ID":"ec96a75e-53f5-41f8-a515-86320f577ba0","Type":"ContainerDied","Data":"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"}
Sep 17 14:21:29.416716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.416645    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm" event={"ID":"ec96a75e-53f5-41f8-a515-86320f577ba0","Type":"ContainerDied","Data":"b041570493666ca0c640ff3a1dc53d9d777cbaaa355cbeba03f92ba4b7113aa6"}
Sep 17 14:21:29.416716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.416673    2570 scope.go:117] "RemoveContainer" containerID="f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"
Sep 17 14:21:29.418115 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.418093    2570 generic.go:356] "Generic (PLEG): container finished" podID="5a0be575-8145-4416-9750-dbf6712cb5bf" containerID="5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f" exitCode=0
Sep 17 14:21:29.418192 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.418138    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"
Sep 17 14:21:29.418192 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.418163    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" event={"ID":"5a0be575-8145-4416-9750-dbf6712cb5bf","Type":"ContainerDied","Data":"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"}
Sep 17 14:21:29.418285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.418206    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn" event={"ID":"5a0be575-8145-4416-9750-dbf6712cb5bf","Type":"ContainerDied","Data":"a106056a647f9e605164f0fa2aee5a42c8f820a82c98e4ca05437950443a9fa5"}
Sep 17 14:21:29.427579 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.427561    2570 scope.go:117] "RemoveContainer" containerID="f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"
Sep 17 14:21:29.427960 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:21:29.427932    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec\": container with ID starting with f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec not found: ID does not exist" containerID="f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"
Sep 17 14:21:29.428076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.427972    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec"} err="failed to get container status \"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec\": rpc error: code = NotFound desc = could not find container \"f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec\": container with ID starting with f045b693324986d26eb8f79f0f28a31f3a5f5c8e89fa55778d4f8dfe4d403bec not found: ID does not exist"
Sep 17 14:21:29.428076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.427996    2570 scope.go:117] "RemoveContainer" containerID="5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"
Sep 17 14:21:29.436830 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.436813    2570 scope.go:117] "RemoveContainer" containerID="5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"
Sep 17 14:21:29.437103 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:21:29.437085    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f\": container with ID starting with 5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f not found: ID does not exist" containerID="5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"
Sep 17 14:21:29.437159 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.437109    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f"} err="failed to get container status \"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f\": rpc error: code = NotFound desc = could not find container \"5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f\": container with ID starting with 5fd2c0a39e7c58b6254f7fe8292d506e74b726663f4fe0876f3435b241823b5f not found: ID does not exist"
Sep 17 14:21:29.443339 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.443317    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:21:29.447687 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.447665    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5cdd59e7-cdbcc798f-l77vm"]
Sep 17 14:21:29.470161 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.470129    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:21:29.488184 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:29.488151    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5cdd59e7-9dc4d4f45-2bccn"]
Sep 17 14:21:30.183028 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:30.182993    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a0be575-8145-4416-9750-dbf6712cb5bf" path="/var/lib/kubelet/pods/5a0be575-8145-4416-9750-dbf6712cb5bf/volumes"
Sep 17 14:21:30.183379 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:30.183345    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec96a75e-53f5-41f8-a515-86320f577ba0" path="/var/lib/kubelet/pods/ec96a75e-53f5-41f8-a515-86320f577ba0/volumes"
Sep 17 14:21:51.416936 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:51.416896    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:21:58.403422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:58.403393    2570 scope.go:117] "RemoveContainer" containerID="c53ebb2a6444f3925c995d3252a91aea5ca8142851661adbffb4143f06d64682"
Sep 17 14:21:58.411654 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:21:58.411633    2570 scope.go:117] "RemoveContainer" containerID="4afe67a6c37b456fef690a414a03259bf5569e61eebbf929d452f876c7a07566"
Sep 17 14:22:26.224715 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:22:26.224679    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:22:52.916269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:22:52.916183    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:23:34.898146 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898108    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898494    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a0be575-8145-4416-9750-dbf6712cb5bf" containerName="maas-api"
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898504    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a0be575-8145-4416-9750-dbf6712cb5bf" containerName="maas-api"
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898530    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec96a75e-53f5-41f8-a515-86320f577ba0" containerName="payload-processing"
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898536    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec96a75e-53f5-41f8-a515-86320f577ba0" containerName="payload-processing"
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898598    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a0be575-8145-4416-9750-dbf6712cb5bf" containerName="maas-api"
Sep 17 14:23:34.898614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.898610    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec96a75e-53f5-41f8-a515-86320f577ba0" containerName="payload-processing"
Sep 17 14:23:34.901332 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.901313    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:34.904421 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.904401    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6fce83-openshift-default-dockercfg-kls67\""
Sep 17 14:23:34.914587 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:34.914562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:23:35.025526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025525    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025640    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025667    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025780    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.025834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025833    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84wd7\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.026058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.026058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.025927    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127279 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127236    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84wd7\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127279 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127278    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127300    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127354    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127422    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127458    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127484    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127862 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.127944 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.127924    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.128047 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.128024    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.128391 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.128072    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.128463 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.128111    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.128543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.128511    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.129933 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.129908    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.130070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.130052    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.136276 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.136255    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.136445 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.136424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84wd7\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7\") pod \"e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") " pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.211892 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.211852    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:35.498453 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.498427    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:23:35.503396 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:23:35.503153    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50564ad9_11a8_452c_bf84_1746292bd529.slice/crio-823e02f41f2f9e4f1c1d73bd0bee1974ca1a6d485b1e238a20fa53418eec4cd7 WatchSource:0}: Error finding container 823e02f41f2f9e4f1c1d73bd0bee1974ca1a6d485b1e238a20fa53418eec4cd7: Status 404 returned error can't find the container with id 823e02f41f2f9e4f1c1d73bd0bee1974ca1a6d485b1e238a20fa53418eec4cd7
Sep 17 14:23:35.505427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.505397    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:23:35.505507 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.505461    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:23:35.505507 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:35.505489    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:23:36.099625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.099590    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerStarted","Data":"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"}
Sep 17 14:23:36.100065 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.099649    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerStarted","Data":"823e02f41f2f9e4f1c1d73bd0bee1974ca1a6d485b1e238a20fa53418eec4cd7"}
Sep 17 14:23:36.121829 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.121778    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podStartSLOduration=2.121763086 podStartE2EDuration="2.121763086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:23:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:23:36.119877682 +0000 UTC m=+1658.356755159" watchObservedRunningTime="2026-09-17 14:23:36.121763086 +0000 UTC m=+1658.358640562"
Sep 17 14:23:36.212847 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.212808    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:36.214404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.214380    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:23:36.214534 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:36.214435    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:23:37.216515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:37.216490    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:38.109593 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:38.109562    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:38.110387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:38.110372    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:23:40.212394 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:40.212363    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:23:44.336161 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.336110    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:23:44.339996 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.339976    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.342921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.342899    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6fce83\""
Sep 17 14:23:44.354450 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.353795    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:23:44.381861 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.381820    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:23:44.385602 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.385580    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.388598 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.388577    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6fce83-dockercfg-4jmnw\""
Sep 17 14:23:44.397061 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.397037    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:23:44.434915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.434881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q58l\" (UniqueName: \"kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.435115 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.434922    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.536252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.536202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.536450 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.536307    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q58l\" (UniqueName: \"kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.536450 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.536419    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.536557 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.536541    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltf7\" (UniqueName: \"kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.537167 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.537147    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.545904 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.545876    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q58l\" (UniqueName: \"kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l\") pod \"payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.637252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.637160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltf7\" (UniqueName: \"kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.637252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.637225    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.637938 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.637919    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.648736 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.648712    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltf7\" (UniqueName: \"kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7\") pod \"payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") " pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.651613 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.651586    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:44.697250 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.697210    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:44.928299 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.928277    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:23:44.929754 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:23:44.929716    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbc1905_f574_4b68_8d7a_4f40056edebb.slice/crio-20257f998d805f323d523bed7a74933d7bb3677deb0ad34fdd278c729fd5d874 WatchSource:0}: Error finding container 20257f998d805f323d523bed7a74933d7bb3677deb0ad34fdd278c729fd5d874: Status 404 returned error can't find the container with id 20257f998d805f323d523bed7a74933d7bb3677deb0ad34fdd278c729fd5d874
Sep 17 14:23:44.990145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:44.990116    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:23:44.991384 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:23:44.991352    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc922a27e_de29_47f2_8e6e_47c1347ed55b.slice/crio-7ec6968da3c5c87d11ee2bbcd0b89aac76ec95f3a84458ab48e580c8e86c4cc2 WatchSource:0}: Error finding container 7ec6968da3c5c87d11ee2bbcd0b89aac76ec95f3a84458ab48e580c8e86c4cc2: Status 404 returned error can't find the container with id 7ec6968da3c5c87d11ee2bbcd0b89aac76ec95f3a84458ab48e580c8e86c4cc2
Sep 17 14:23:45.148194 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.148153    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" event={"ID":"c922a27e-de29-47f2-8e6e-47c1347ed55b","Type":"ContainerStarted","Data":"f15a3e396e8815bf3449baf821f774740ad54872cf96030ac1def9efd1dd1b17"}
Sep 17 14:23:45.148398 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.148217    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" event={"ID":"c922a27e-de29-47f2-8e6e-47c1347ed55b","Type":"ContainerStarted","Data":"7ec6968da3c5c87d11ee2bbcd0b89aac76ec95f3a84458ab48e580c8e86c4cc2"}
Sep 17 14:23:45.148398 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.148244    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:45.149898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.149856    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" event={"ID":"0bbc1905-f574-4b68-8d7a-4f40056edebb","Type":"ContainerStarted","Data":"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"}
Sep 17 14:23:45.150044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.149908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" event={"ID":"0bbc1905-f574-4b68-8d7a-4f40056edebb","Type":"ContainerStarted","Data":"20257f998d805f323d523bed7a74933d7bb3677deb0ad34fdd278c729fd5d874"}
Sep 17 14:23:45.150044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.149950    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:45.168056 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.167989    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" podStartSLOduration=1.167967273 podStartE2EDuration="1.167967273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:23:45.16670557 +0000 UTC m=+1667.403583051" watchObservedRunningTime="2026-09-17 14:23:45.167967273 +0000 UTC m=+1667.404844751"
Sep 17 14:23:45.189501 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:45.189394    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" podStartSLOduration=1.1893711630000001 podStartE2EDuration="1.189371163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:23:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:23:45.186014984 +0000 UTC m=+1667.422892462" watchObservedRunningTime="2026-09-17 14:23:45.189371163 +0000 UTC m=+1667.426248643"
Sep 17 14:23:56.160440 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:56.160410    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:23:56.160932 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:56.160490    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:23:58.233725 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:58.233690    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerID="403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292" exitCode=137
Sep 17 14:23:58.234130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:58.233765    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerDied","Data":"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"}
Sep 17 14:23:58.234219 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:58.234153    2570 scope.go:117] "RemoveContainer" containerID="403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"
Sep 17 14:23:59.241008 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:59.240973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerStarted","Data":"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"}
Sep 17 14:23:59.241504 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:23:59.241196    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:24:10.249295 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:10.249268    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:24:16.728254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.728214    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:24:16.731965 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.731948    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.734680 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.734658    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a77c87-dockercfg-zsjmx\""
Sep 17 14:24:16.734816 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.734782    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a77c87\""
Sep 17 14:24:16.742196 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.742174    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:24:16.848094 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.848062    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.848266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.848163    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7knx\" (UniqueName: \"kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.949136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.949103    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7knx\" (UniqueName: \"kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.949310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.949183    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.949875 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.949842    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:16.958406 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:16.958379    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7knx\" (UniqueName: \"kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx\") pod \"payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") " pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:17.043324 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:17.043252    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:17.325476 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:17.325446    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:24:17.326994 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:24:17.326965    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c259d2_ea17_4a75_afa8_8db691845ecc.slice/crio-1ec728032e2f2789a0b5aadae0ae814ae9b43e69fb9f868560e1f70872086ee9 WatchSource:0}: Error finding container 1ec728032e2f2789a0b5aadae0ae814ae9b43e69fb9f868560e1f70872086ee9: Status 404 returned error can't find the container with id 1ec728032e2f2789a0b5aadae0ae814ae9b43e69fb9f868560e1f70872086ee9
Sep 17 14:24:17.347788 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:17.347762    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" event={"ID":"98c259d2-ea17-4a75-afa8-8db691845ecc","Type":"ContainerStarted","Data":"1ec728032e2f2789a0b5aadae0ae814ae9b43e69fb9f868560e1f70872086ee9"}
Sep 17 14:24:18.357452 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:18.357409    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" event={"ID":"98c259d2-ea17-4a75-afa8-8db691845ecc","Type":"ContainerStarted","Data":"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"}
Sep 17 14:24:18.357975 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:18.357504    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:18.376726 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:18.376684    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" podStartSLOduration=2.376665059 podStartE2EDuration="2.376665059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:24:18.376498795 +0000 UTC m=+1700.613376274" watchObservedRunningTime="2026-09-17 14:24:18.376665059 +0000 UTC m=+1700.613542536"
Sep 17 14:24:29.364867 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:29.364838    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:24:31.434973 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:31.434939    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:24:43.507951 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.507858    2570 generic.go:356] "Generic (PLEG): container finished" podID="50564ad9-11a8-452c-bf84-1746292bd529" containerID="92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc" exitCode=0
Sep 17 14:24:43.508446 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.507945    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerDied","Data":"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"}
Sep 17 14:24:43.508517 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.508502    2570 scope.go:117] "RemoveContainer" containerID="92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"
Sep 17 14:24:43.510005 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.509972    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:24:43.510128 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.510061    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:24:43.510128 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:43.510113    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 17 14:24:44.516776 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:44.516717    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerStarted","Data":"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"}
Sep 17 14:24:44.842210 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:44.842122    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:24:45.212473 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:45.212443    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:24:45.212710 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:45.212691    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:24:45.212781 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:45.212758    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:24:46.212494 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:46.212457    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:24:46.212918 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:46.212512    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:24:47.212375 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:47.212341    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:24:47.212592 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:47.212395    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:24:48.212756 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:48.212695    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:24:48.213124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:48.212783    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:24:49.212537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:49.212491    2570 patch_prober.go:28] interesting pod/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 17 14:24:49.212710 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:49.212569    2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 17 14:24:50.271753 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:50.271701    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:24:50.272275 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:50.272250    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:24:50.273310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:50.273181    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:24:59.889520 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:24:59.889456    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" podUID="da07655d-388b-410d-b755-9b8d755d834e" containerName="main" containerID="cri-o://113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20" gracePeriod=45
Sep 17 14:25:00.146511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.146444    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:25:00.210101 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210058    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvzzt\" (UniqueName: \"kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210290 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210182    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210290 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210211    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210290 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210239    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210301    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210331    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir\") pod \"da07655d-388b-410d-b755-9b8d755d834e\" (UID: \"da07655d-388b-410d-b755-9b8d755d834e\") "
Sep 17 14:25:00.210574 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210552    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache" (OuterVolumeSpecName: "model-cache") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:00.210787 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210766    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.210951 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210782    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home" (OuterVolumeSpecName: "home") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:00.211080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.210892    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:00.212767 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.212711    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt" (OuterVolumeSpecName: "kube-api-access-qvzzt") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "kube-api-access-qvzzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:00.213603 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.213572    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm" (OuterVolumeSpecName: "dshm") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:00.213693 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.213668    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "da07655d-388b-410d-b755-9b8d755d834e" (UID: "da07655d-388b-410d-b755-9b8d755d834e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:25:00.312248 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.312217    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.312248 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.312249    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.312447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.312295    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvzzt\" (UniqueName: \"kubernetes.io/projected/da07655d-388b-410d-b755-9b8d755d834e-kube-api-access-qvzzt\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.312447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.312304    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da07655d-388b-410d-b755-9b8d755d834e-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.312447 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.312313    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da07655d-388b-410d-b755-9b8d755d834e-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:00.618220 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.618188    2570 generic.go:356] "Generic (PLEG): container finished" podID="da07655d-388b-410d-b755-9b8d755d834e" containerID="113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20" exitCode=0
Sep 17 14:25:00.618419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.618246    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"
Sep 17 14:25:00.618419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.618256    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" event={"ID":"da07655d-388b-410d-b755-9b8d755d834e","Type":"ContainerDied","Data":"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"}
Sep 17 14:25:00.618419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.618297    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt" event={"ID":"da07655d-388b-410d-b755-9b8d755d834e","Type":"ContainerDied","Data":"6e6f6c8cb9a71779a2f0d47510b0f9547b2e6eb89c85cd74adaa59abca33510f"}
Sep 17 14:25:00.618419 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.618315    2570 scope.go:117] "RemoveContainer" containerID="113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"
Sep 17 14:25:00.628452 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.628429    2570 scope.go:117] "RemoveContainer" containerID="113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"
Sep 17 14:25:00.628726 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:00.628709    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20\": container with ID starting with 113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20 not found: ID does not exist" containerID="113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"
Sep 17 14:25:00.628833 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.628752    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20"} err="failed to get container status \"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20\": rpc error: code = NotFound desc = could not find container \"113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20\": container with ID starting with 113d85ffad3c2f6707c69331bd9798351558db78bf3c69faa55b7d702f150d20 not found: ID does not exist"
Sep 17 14:25:00.657778 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.657748    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:25:00.663334 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:00.663296    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-d9b17d/simulator-w0-d9b17d-kserve-7445d774c4-frvdt"]
Sep 17 14:25:02.182198 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.182163    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da07655d-388b-410d-b755-9b8d755d834e" path="/var/lib/kubelet/pods/da07655d-388b-410d-b755-9b8d755d834e/volumes"
Sep 17 14:25:02.509444 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.509360    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:25:02.509646 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.509623    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" podUID="c922a27e-de29-47f2-8e6e-47c1347ed55b" containerName="payload-processing" containerID="cri-o://f15a3e396e8815bf3449baf821f774740ad54872cf96030ac1def9efd1dd1b17" gracePeriod=30
Sep 17 14:25:02.518588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.517917    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:25:02.518588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.518171    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" podUID="0bbc1905-f574-4b68-8d7a-4f40056edebb" containerName="payload-pre-processing" containerID="cri-o://bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7" gracePeriod=30
Sep 17 14:25:02.632250 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.632213    2570 generic.go:356] "Generic (PLEG): container finished" podID="c922a27e-de29-47f2-8e6e-47c1347ed55b" containerID="f15a3e396e8815bf3449baf821f774740ad54872cf96030ac1def9efd1dd1b17" exitCode=0
Sep 17 14:25:02.632380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.632291    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" event={"ID":"c922a27e-de29-47f2-8e6e-47c1347ed55b","Type":"ContainerDied","Data":"f15a3e396e8815bf3449baf821f774740ad54872cf96030ac1def9efd1dd1b17"}
Sep 17 14:25:02.800952 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.800919    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:25:02.827403 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.827382    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:25:02.953470 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.953425    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ltf7\" (UniqueName: \"kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7\") pod \"c922a27e-de29-47f2-8e6e-47c1347ed55b\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") "
Sep 17 14:25:02.953662 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.953517    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q58l\" (UniqueName: \"kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l\") pod \"0bbc1905-f574-4b68-8d7a-4f40056edebb\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") "
Sep 17 14:25:02.953662 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.953577    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume\") pod \"0bbc1905-f574-4b68-8d7a-4f40056edebb\" (UID: \"0bbc1905-f574-4b68-8d7a-4f40056edebb\") "
Sep 17 14:25:02.953828 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.953686    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume\") pod \"c922a27e-de29-47f2-8e6e-47c1347ed55b\" (UID: \"c922a27e-de29-47f2-8e6e-47c1347ed55b\") "
Sep 17 14:25:02.954046 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.954016    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bbc1905-f574-4b68-8d7a-4f40056edebb" (UID: "0bbc1905-f574-4b68-8d7a-4f40056edebb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:02.954121 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.954078    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c922a27e-de29-47f2-8e6e-47c1347ed55b" (UID: "c922a27e-de29-47f2-8e6e-47c1347ed55b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:02.955922 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.955890    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7" (OuterVolumeSpecName: "kube-api-access-9ltf7") pod "c922a27e-de29-47f2-8e6e-47c1347ed55b" (UID: "c922a27e-de29-47f2-8e6e-47c1347ed55b"). InnerVolumeSpecName "kube-api-access-9ltf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:02.956263 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:02.956240    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l" (OuterVolumeSpecName: "kube-api-access-9q58l") pod "0bbc1905-f574-4b68-8d7a-4f40056edebb" (UID: "0bbc1905-f574-4b68-8d7a-4f40056edebb"). InnerVolumeSpecName "kube-api-access-9q58l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:03.055526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.055427    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c922a27e-de29-47f2-8e6e-47c1347ed55b-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:03.055526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.055483    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ltf7\" (UniqueName: \"kubernetes.io/projected/c922a27e-de29-47f2-8e6e-47c1347ed55b-kube-api-access-9ltf7\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:03.055526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.055499    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q58l\" (UniqueName: \"kubernetes.io/projected/0bbc1905-f574-4b68-8d7a-4f40056edebb-kube-api-access-9q58l\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:03.055526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.055512    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bbc1905-f574-4b68-8d7a-4f40056edebb-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:03.641161 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.641131    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"
Sep 17 14:25:03.641597 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.641166    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg" event={"ID":"c922a27e-de29-47f2-8e6e-47c1347ed55b","Type":"ContainerDied","Data":"7ec6968da3c5c87d11ee2bbcd0b89aac76ec95f3a84458ab48e580c8e86c4cc2"}
Sep 17 14:25:03.641597 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.641234    2570 scope.go:117] "RemoveContainer" containerID="f15a3e396e8815bf3449baf821f774740ad54872cf96030ac1def9efd1dd1b17"
Sep 17 14:25:03.642723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.642701    2570 generic.go:356] "Generic (PLEG): container finished" podID="0bbc1905-f574-4b68-8d7a-4f40056edebb" containerID="bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7" exitCode=0
Sep 17 14:25:03.642890 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.642875    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"
Sep 17 14:25:03.642994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.642872    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" event={"ID":"0bbc1905-f574-4b68-8d7a-4f40056edebb","Type":"ContainerDied","Data":"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"}
Sep 17 14:25:03.643050 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.643011    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d" event={"ID":"0bbc1905-f574-4b68-8d7a-4f40056edebb","Type":"ContainerDied","Data":"20257f998d805f323d523bed7a74933d7bb3677deb0ad34fdd278c729fd5d874"}
Sep 17 14:25:03.650909 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.650886    2570 scope.go:117] "RemoveContainer" containerID="bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"
Sep 17 14:25:03.662503 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.662485    2570 scope.go:117] "RemoveContainer" containerID="bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"
Sep 17 14:25:03.662819 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:03.662801    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7\": container with ID starting with bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7 not found: ID does not exist" containerID="bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"
Sep 17 14:25:03.662876 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.662828    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7"} err="failed to get container status \"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7\": rpc error: code = NotFound desc = could not find container \"bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7\": container with ID starting with bf3e9b1d1747b9301ef9ab1b614fe4fbc370d47031cfeb271f7b38d76379a6d7 not found: ID does not exist"
Sep 17 14:25:03.669233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.669209    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:25:03.674937 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.674916    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6fce83-6ffcf884fc-phjkg"]
Sep 17 14:25:03.688860 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.688840    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:25:03.692863 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:03.692844    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6fce83-756cb4ff86-h5f5d"]
Sep 17 14:25:04.184443 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:04.184407    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbc1905-f574-4b68-8d7a-4f40056edebb" path="/var/lib/kubelet/pods/0bbc1905-f574-4b68-8d7a-4f40056edebb/volumes"
Sep 17 14:25:04.184821 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:04.184808    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c922a27e-de29-47f2-8e6e-47c1347ed55b" path="/var/lib/kubelet/pods/c922a27e-de29-47f2-8e6e-47c1347ed55b/volumes"
Sep 17 14:25:10.631162 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:10.631127    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:25:10.631626 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:10.631333    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy" containerID="cri-o://510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d" gracePeriod=30
Sep 17 14:25:15.893379 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.893323    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:25:15.984102 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984070    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984102 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984107    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984135    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984153    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984173    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984198    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984278    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84wd7\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984345 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984333    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984360    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket\") pod \"50564ad9-11a8-452c-bf84-1746292bd529\" (UID: \"50564ad9-11a8-452c-bf84-1746292bd529\") "
Sep 17 14:25:15.984678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984526    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:15.984678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984545    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:15.984678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984565    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:15.984898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984701    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:15.984898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984715    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data" (OuterVolumeSpecName: "istio-data") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:15.984979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984941    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:15.984979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984964    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:15.985099 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984977    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50564ad9-11a8-452c-bf84-1746292bd529-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:15.985099 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.984991    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:15.985099 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.985003    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:15.986636 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.986595    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token" (OuterVolumeSpecName: "istio-token") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:15.986815 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.986658    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:25:15.987063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.987040    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:25:15.987157 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:15.987067    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7" (OuterVolumeSpecName: "kube-api-access-84wd7") pod "50564ad9-11a8-452c-bf84-1746292bd529" (UID: "50564ad9-11a8-452c-bf84-1746292bd529"). InnerVolumeSpecName "kube-api-access-84wd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:16.086259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.086224    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50564ad9-11a8-452c-bf84-1746292bd529-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:16.086259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.086254    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:16.086461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.086268    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84wd7\" (UniqueName: \"kubernetes.io/projected/50564ad9-11a8-452c-bf84-1746292bd529-kube-api-access-84wd7\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:16.086461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.086280    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50564ad9-11a8-452c-bf84-1746292bd529-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:16.717165 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.717132    2570 generic.go:356] "Generic (PLEG): container finished" podID="50564ad9-11a8-452c-bf84-1746292bd529" containerID="510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d" exitCode=0
Sep 17 14:25:16.717325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.717181    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"
Sep 17 14:25:16.717325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.717199    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerDied","Data":"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"}
Sep 17 14:25:16.717325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.717242    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2" event={"ID":"50564ad9-11a8-452c-bf84-1746292bd529","Type":"ContainerDied","Data":"823e02f41f2f9e4f1c1d73bd0bee1974ca1a6d485b1e238a20fa53418eec4cd7"}
Sep 17 14:25:16.717325 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.717264    2570 scope.go:117] "RemoveContainer" containerID="510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"
Sep 17 14:25:16.726621 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.726602    2570 scope.go:117] "RemoveContainer" containerID="92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"
Sep 17 14:25:16.735478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.735440    2570 scope.go:117] "RemoveContainer" containerID="510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"
Sep 17 14:25:16.735816 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:16.735796    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d\": container with ID starting with 510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d not found: ID does not exist" containerID="510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"
Sep 17 14:25:16.735874 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.735826    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d"} err="failed to get container status \"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d\": rpc error: code = NotFound desc = could not find container \"510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d\": container with ID starting with 510691e41f292c76ef0da8eb9e907ef5207d325e95206d9def803f383ebd623d not found: ID does not exist"
Sep 17 14:25:16.735874 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.735844    2570 scope.go:117] "RemoveContainer" containerID="92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"
Sep 17 14:25:16.736119 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:16.736098    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc\": container with ID starting with 92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc not found: ID does not exist" containerID="92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"
Sep 17 14:25:16.736174 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.736131    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc"} err="failed to get container status \"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc\": rpc error: code = NotFound desc = could not find container \"92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc\": container with ID starting with 92990fb00f13eb96992f026214cc9cc5aae6bbf8689f53fa4bf9a63b66f076dc not found: ID does not exist"
Sep 17 14:25:16.740572 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.740546    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:25:16.748671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:16.748651    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6fce83-openshift-default-5658884b46-xp7v2"]
Sep 17 14:25:18.183743 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:18.183693    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50564ad9-11a8-452c-bf84-1746292bd529" path="/var/lib/kubelet/pods/50564ad9-11a8-452c-bf84-1746292bd529/volumes"
Sep 17 14:25:19.213074 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:19.213035    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:25:27.702134 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:27.702101    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:25:27.702598 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:27.702321    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" podUID="98c259d2-ea17-4a75-afa8-8db691845ecc" containerName="payload-processing" containerID="cri-o://7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c" gracePeriod=30
Sep 17 14:25:27.966891 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:27.966548    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:25:28.102633 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.102601    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7knx\" (UniqueName: \"kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx\") pod \"98c259d2-ea17-4a75-afa8-8db691845ecc\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") "
Sep 17 14:25:28.102824 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.102674    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume\") pod \"98c259d2-ea17-4a75-afa8-8db691845ecc\" (UID: \"98c259d2-ea17-4a75-afa8-8db691845ecc\") "
Sep 17 14:25:28.103079 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.103057    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "98c259d2-ea17-4a75-afa8-8db691845ecc" (UID: "98c259d2-ea17-4a75-afa8-8db691845ecc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:28.104853 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.104830    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx" (OuterVolumeSpecName: "kube-api-access-p7knx") pod "98c259d2-ea17-4a75-afa8-8db691845ecc" (UID: "98c259d2-ea17-4a75-afa8-8db691845ecc"). InnerVolumeSpecName "kube-api-access-p7knx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:28.204123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.204095    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7knx\" (UniqueName: \"kubernetes.io/projected/98c259d2-ea17-4a75-afa8-8db691845ecc-kube-api-access-p7knx\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:28.204123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.204126    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/98c259d2-ea17-4a75-afa8-8db691845ecc-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:28.784609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.784573    2570 generic.go:356] "Generic (PLEG): container finished" podID="98c259d2-ea17-4a75-afa8-8db691845ecc" containerID="7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c" exitCode=0
Sep 17 14:25:28.784609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.784612    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"
Sep 17 14:25:28.785107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.784666    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" event={"ID":"98c259d2-ea17-4a75-afa8-8db691845ecc","Type":"ContainerDied","Data":"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"}
Sep 17 14:25:28.785107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.784720    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h" event={"ID":"98c259d2-ea17-4a75-afa8-8db691845ecc","Type":"ContainerDied","Data":"1ec728032e2f2789a0b5aadae0ae814ae9b43e69fb9f868560e1f70872086ee9"}
Sep 17 14:25:28.785107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.784762    2570 scope.go:117] "RemoveContainer" containerID="7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"
Sep 17 14:25:28.793888 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.793871    2570 scope.go:117] "RemoveContainer" containerID="7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"
Sep 17 14:25:28.794150 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:28.794133    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c\": container with ID starting with 7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c not found: ID does not exist" containerID="7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"
Sep 17 14:25:28.794201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.794159    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c"} err="failed to get container status \"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c\": rpc error: code = NotFound desc = could not find container \"7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c\": container with ID starting with 7c79b1a1fdd1f98e2ff7e0d4a46f62852efbffd84f5001b4cd7e9327c2c4517c not found: ID does not exist"
Sep 17 14:25:28.800848 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.800826    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:25:28.804750 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:28.804713    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a77c87-79dc65666c-fcv9h"]
Sep 17 14:25:30.182593 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:30.182563    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c259d2-ea17-4a75-afa8-8db691845ecc" path="/var/lib/kubelet/pods/98c259d2-ea17-4a75-afa8-8db691845ecc/volumes"
Sep 17 14:25:46.170846 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.170812    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:25:46.171324 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.171088    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" podUID="7059ca5d-dd99-454a-9216-86af0c7cbd96" containerName="payload-processing" containerID="cri-o://7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f" gracePeriod=30
Sep 17 14:25:46.424395 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.424340    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:25:46.586981 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.586952    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume\") pod \"7059ca5d-dd99-454a-9216-86af0c7cbd96\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") "
Sep 17 14:25:46.587143 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.587065    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwkgt\" (UniqueName: \"kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt\") pod \"7059ca5d-dd99-454a-9216-86af0c7cbd96\" (UID: \"7059ca5d-dd99-454a-9216-86af0c7cbd96\") "
Sep 17 14:25:46.587282 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.587258    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7059ca5d-dd99-454a-9216-86af0c7cbd96" (UID: "7059ca5d-dd99-454a-9216-86af0c7cbd96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:25:46.589201 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.589176    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt" (OuterVolumeSpecName: "kube-api-access-nwkgt") pod "7059ca5d-dd99-454a-9216-86af0c7cbd96" (UID: "7059ca5d-dd99-454a-9216-86af0c7cbd96"). InnerVolumeSpecName "kube-api-access-nwkgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:25:46.688627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.688547    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwkgt\" (UniqueName: \"kubernetes.io/projected/7059ca5d-dd99-454a-9216-86af0c7cbd96-kube-api-access-nwkgt\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:46.688627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.688574    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7059ca5d-dd99-454a-9216-86af0c7cbd96-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:25:46.885166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.885133    2570 generic.go:356] "Generic (PLEG): container finished" podID="7059ca5d-dd99-454a-9216-86af0c7cbd96" containerID="7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f" exitCode=0
Sep 17 14:25:46.885316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.885176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" event={"ID":"7059ca5d-dd99-454a-9216-86af0c7cbd96","Type":"ContainerDied","Data":"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"}
Sep 17 14:25:46.885316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.885177    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"
Sep 17 14:25:46.885316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.885217    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk" event={"ID":"7059ca5d-dd99-454a-9216-86af0c7cbd96","Type":"ContainerDied","Data":"02a1ec0bd338f8d257d31acc442b4951c46d23015f224b8772d2ab0da53cd2a8"}
Sep 17 14:25:46.885316 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.885238    2570 scope.go:117] "RemoveContainer" containerID="7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"
Sep 17 14:25:46.896933 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.896902    2570 scope.go:117] "RemoveContainer" containerID="7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"
Sep 17 14:25:46.897396 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:25:46.897373    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f\": container with ID starting with 7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f not found: ID does not exist" containerID="7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"
Sep 17 14:25:46.897500 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.897408    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f"} err="failed to get container status \"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f\": rpc error: code = NotFound desc = could not find container \"7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f\": container with ID starting with 7e884e4f57f89533fc280b2392134a08659b7eb170e143e8d742d298a097159f not found: ID does not exist"
Sep 17 14:25:46.910593 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.910568    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:25:46.914867 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:46.914847    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2b8275-5969945d85-dz2mk"]
Sep 17 14:25:48.182019 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:48.181987    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7059ca5d-dd99-454a-9216-86af0c7cbd96" path="/var/lib/kubelet/pods/7059ca5d-dd99-454a-9216-86af0c7cbd96/volumes"
Sep 17 14:25:52.908022 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:25:52.907928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:26:05.735640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.735606    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:26:05.736123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.735866    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" podUID="22ff4d3c-4164-4965-975e-2bcb0457f594" containerName="payload-processing" containerID="cri-o://9afd5a34f7b4749afc48a8659e8c9eaaf2f74cda13ce8596f3f472c2ae56f32e" gracePeriod=30
Sep 17 14:26:05.989834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.989753    2570 generic.go:356] "Generic (PLEG): container finished" podID="22ff4d3c-4164-4965-975e-2bcb0457f594" containerID="9afd5a34f7b4749afc48a8659e8c9eaaf2f74cda13ce8596f3f472c2ae56f32e" exitCode=0
Sep 17 14:26:05.989834 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.989767    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" event={"ID":"22ff4d3c-4164-4965-975e-2bcb0457f594","Type":"ContainerDied","Data":"9afd5a34f7b4749afc48a8659e8c9eaaf2f74cda13ce8596f3f472c2ae56f32e"}
Sep 17 14:26:05.990013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.989832    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484" event={"ID":"22ff4d3c-4164-4965-975e-2bcb0457f594","Type":"ContainerDied","Data":"88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d"}
Sep 17 14:26:05.990013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.989857    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88ff7c3849a43012e73b26c5217e67646952bca38cc5c45964485aafff2d1b6d"
Sep 17 14:26:05.991274 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:05.991257    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:26:06.056368 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.056319    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d9mr\" (UniqueName: \"kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr\") pod \"22ff4d3c-4164-4965-975e-2bcb0457f594\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") "
Sep 17 14:26:06.056549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.056423    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume\") pod \"22ff4d3c-4164-4965-975e-2bcb0457f594\" (UID: \"22ff4d3c-4164-4965-975e-2bcb0457f594\") "
Sep 17 14:26:06.056888 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.056847    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "22ff4d3c-4164-4965-975e-2bcb0457f594" (UID: "22ff4d3c-4164-4965-975e-2bcb0457f594"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:06.058639 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.058608    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr" (OuterVolumeSpecName: "kube-api-access-8d9mr") pod "22ff4d3c-4164-4965-975e-2bcb0457f594" (UID: "22ff4d3c-4164-4965-975e-2bcb0457f594"). InnerVolumeSpecName "kube-api-access-8d9mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:06.157528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.157493    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d9mr\" (UniqueName: \"kubernetes.io/projected/22ff4d3c-4164-4965-975e-2bcb0457f594-kube-api-access-8d9mr\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:06.157528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.157520    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/22ff4d3c-4164-4965-975e-2bcb0457f594-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:06.995400 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:06.995367    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"
Sep 17 14:26:07.012195 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:07.012127    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:26:07.018123 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:07.018100    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6e2bb9-8569f6c4c9-jq484"]
Sep 17 14:26:08.182032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:08.182001    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ff4d3c-4164-4965-975e-2bcb0457f594" path="/var/lib/kubelet/pods/22ff4d3c-4164-4965-975e-2bcb0457f594/volumes"
Sep 17 14:26:16.113272 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:16.113237    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:26:17.819664 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:17.819633    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:26:17.829438 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:17.829401    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:26:32.839921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:32.839864    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" podUID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" containerName="main" containerID="cri-o://765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1" gracePeriod=45
Sep 17 14:26:32.851800 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:32.851721    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" podUID="ec571499-5c08-4034-a391-164356db00c1" containerName="main" containerID="cri-o://110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded" gracePeriod=45
Sep 17 14:26:33.102188 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.102133    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:26:33.129673 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.129643    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:26:33.133261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.133232    2570 generic.go:356] "Generic (PLEG): container finished" podID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" containerID="765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1" exitCode=0
Sep 17 14:26:33.133388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.133266    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"
Sep 17 14:26:33.133388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.133314    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" event={"ID":"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7","Type":"ContainerDied","Data":"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"}
Sep 17 14:26:33.133388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.133352    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd" event={"ID":"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7","Type":"ContainerDied","Data":"232bd8255581862c55730e808cf0077f4dde50ec3bc5b7fe1f71f151f9ef9900"}
Sep 17 14:26:33.133388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.133376    2570 scope.go:117] "RemoveContainer" containerID="765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"
Sep 17 14:26:33.135207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.135150    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec571499-5c08-4034-a391-164356db00c1" containerID="110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded" exitCode=0
Sep 17 14:26:33.135207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.135179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" event={"ID":"ec571499-5c08-4034-a391-164356db00c1","Type":"ContainerDied","Data":"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"}
Sep 17 14:26:33.135366 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.135210    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"
Sep 17 14:26:33.135366 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.135229    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz" event={"ID":"ec571499-5c08-4034-a391-164356db00c1","Type":"ContainerDied","Data":"1950f4e3c1137425ce5eaeee3b19ffbceff49a0a7965906c23dab9eb318bc662"}
Sep 17 14:26:33.143679 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.143661    2570 scope.go:117] "RemoveContainer" containerID="765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"
Sep 17 14:26:33.144046 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:26:33.144023    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1\": container with ID starting with 765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1 not found: ID does not exist" containerID="765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"
Sep 17 14:26:33.144142 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.144058    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1"} err="failed to get container status \"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1\": rpc error: code = NotFound desc = could not find container \"765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1\": container with ID starting with 765d8406fb14cfe7a36d93bf3f6fefd18778febf3449a5f6ac8663d3f1a0fdc1 not found: ID does not exist"
Sep 17 14:26:33.144142 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.144076    2570 scope.go:117] "RemoveContainer" containerID="110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"
Sep 17 14:26:33.153896 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.153629    2570 scope.go:117] "RemoveContainer" containerID="110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"
Sep 17 14:26:33.154359 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:26:33.154282    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded\": container with ID starting with 110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded not found: ID does not exist" containerID="110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"
Sep 17 14:26:33.154359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.154315    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded"} err="failed to get container status \"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded\": rpc error: code = NotFound desc = could not find container \"110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded\": container with ID starting with 110491cbcf230341b8cf7cb5e54effe4587201a82d32b8c81d86b9db0abedded not found: ID does not exist"
Sep 17 14:26:33.216850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.216822    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.216860    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.216893    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.216949    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.216980    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt6w5\" (UniqueName: \"kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217011    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217042    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217080    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217102    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217127    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217141    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217233 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217139    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home" (OuterVolumeSpecName: "home") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.217556 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217264    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97h96\" (UniqueName: \"kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96\") pod \"ec571499-5c08-4034-a391-164356db00c1\" (UID: \"ec571499-5c08-4034-a391-164356db00c1\") "
Sep 17 14:26:33.217556 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217332    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir\") pod \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\" (UID: \"6afa62d0-95db-4615-a6bf-ab2bc5b5eee7\") "
Sep 17 14:26:33.217556 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217350    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache" (OuterVolumeSpecName: "model-cache") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.217556 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217528    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.217833 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217574    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home" (OuterVolumeSpecName: "home") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.217833 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217770    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.217943 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217837    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.218166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217946    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.218166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217974    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.218166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217976    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.218166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.217986    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.219792 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.219766    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5" (OuterVolumeSpecName: "kube-api-access-mt6w5") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "kube-api-access-mt6w5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:33.219792 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.219770    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:26:33.219950 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.219839    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm" (OuterVolumeSpecName: "dshm") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.220683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.220566    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm" (OuterVolumeSpecName: "dshm") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:33.220683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.220649    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" (UID: "6afa62d0-95db-4615-a6bf-ab2bc5b5eee7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:26:33.220683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.220655    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96" (OuterVolumeSpecName: "kube-api-access-97h96") pod "ec571499-5c08-4034-a391-164356db00c1" (UID: "ec571499-5c08-4034-a391-164356db00c1"). InnerVolumeSpecName "kube-api-access-97h96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:33.318478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318446    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt6w5\" (UniqueName: \"kubernetes.io/projected/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-kube-api-access-mt6w5\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318476    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318485    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec571499-5c08-4034-a391-164356db00c1-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318494    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97h96\" (UniqueName: \"kubernetes.io/projected/ec571499-5c08-4034-a391-164356db00c1-kube-api-access-97h96\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318504    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318513    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec571499-5c08-4034-a391-164356db00c1-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.318713 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.318520    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:33.486919 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.486886    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:26:33.494134 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.494110    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/premium-w4-079b38-kserve-7cc97d947-p5gvd"]
Sep 17 14:26:33.514386 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.514361    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:26:33.522978 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:33.522955    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-079b38/simulator-w4-079b38-kserve-764b944566-p9ztz"]
Sep 17 14:26:34.181411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:34.181376    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" path="/var/lib/kubelet/pods/6afa62d0-95db-4615-a6bf-ab2bc5b5eee7/volumes"
Sep 17 14:26:34.181798 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:34.181778    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec571499-5c08-4034-a391-164356db00c1" path="/var/lib/kubelet/pods/ec571499-5c08-4034-a391-164356db00c1/volumes"
Sep 17 14:26:37.316966 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:37.316940    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:26:37.317355 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:37.317135    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy" containerID="cri-o://9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657" gracePeriod=30
Sep 17 14:26:42.322753 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.322693    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:26:42.575917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.575848    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:26:42.595869 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.595839    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.595876    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.595894    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.595931    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.595968    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596011    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596035    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596393 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596077    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnltn\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596393 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596109    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data\") pod \"982631a4-7721-44f1-9032-377754e77c64\" (UID: \"982631a4-7721-44f1-9032-377754e77c64\") "
Sep 17 14:26:42.596393 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596232    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:42.596393 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596359    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:26:42.596811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596428    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data" (OuterVolumeSpecName: "istio-data") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:42.596811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596437    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.596811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596608    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:42.596811 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.596623    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:42.598310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.598277    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token" (OuterVolumeSpecName: "istio-token") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:42.598414 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.598371    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:26:42.598581 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.598556    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:26:42.598850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.598833    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn" (OuterVolumeSpecName: "kube-api-access-cnltn") pod "982631a4-7721-44f1-9032-377754e77c64" (UID: "982631a4-7721-44f1-9032-377754e77c64"). InnerVolumeSpecName "kube-api-access-cnltn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:26:42.697678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697645    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697673    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697685    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnltn\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-kube-api-access-cnltn\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697693    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697702    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/982631a4-7721-44f1-9032-377754e77c64-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697709    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/982631a4-7721-44f1-9032-377754e77c64-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697717    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/982631a4-7721-44f1-9032-377754e77c64-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:42.697931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:42.697724    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/982631a4-7721-44f1-9032-377754e77c64-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:26:43.188890 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.188848    2570 generic.go:356] "Generic (PLEG): container finished" podID="982631a4-7721-44f1-9032-377754e77c64" containerID="9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657" exitCode=0
Sep 17 14:26:43.189073 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.188908    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerDied","Data":"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"}
Sep 17 14:26:43.189073 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.188985    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949" event={"ID":"982631a4-7721-44f1-9032-377754e77c64","Type":"ContainerDied","Data":"ee7954015621a7e389f0d081f74621f15efbcb8d8968ca878297b3912e7171ce"}
Sep 17 14:26:43.189073 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.188922    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"
Sep 17 14:26:43.189073 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.189016    2570 scope.go:117] "RemoveContainer" containerID="9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"
Sep 17 14:26:43.199864 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.199847    2570 scope.go:117] "RemoveContainer" containerID="6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"
Sep 17 14:26:43.209261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.209238    2570 scope.go:117] "RemoveContainer" containerID="9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"
Sep 17 14:26:43.209579 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:26:43.209555    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657\": container with ID starting with 9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657 not found: ID does not exist" containerID="9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"
Sep 17 14:26:43.209649 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.209595    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657"} err="failed to get container status \"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657\": rpc error: code = NotFound desc = could not find container \"9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657\": container with ID starting with 9e3e0431181028cfd057fa60c84839ba13de2548a4d093aa83af5177b6019657 not found: ID does not exist"
Sep 17 14:26:43.209649 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.209621    2570 scope.go:117] "RemoveContainer" containerID="6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"
Sep 17 14:26:43.210104 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:26:43.210083    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037\": container with ID starting with 6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037 not found: ID does not exist" containerID="6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"
Sep 17 14:26:43.210209 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.210108    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037"} err="failed to get container status \"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037\": rpc error: code = NotFound desc = could not find container \"6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037\": container with ID starting with 6b69fa0db50b31b27f910a7b8f237c5a756b9de80126924d121d9cdc0903f037 not found: ID does not exist"
Sep 17 14:26:43.213133 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.213112    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:26:43.217446 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:43.217426    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-079b38-openshift-default-778d985f94-q6949"]
Sep 17 14:26:44.182026 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:44.181986    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="982631a4-7721-44f1-9032-377754e77c64" path="/var/lib/kubelet/pods/982631a4-7721-44f1-9032-377754e77c64/volumes"
Sep 17 14:26:47.797661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:47.797626    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:26:58.495119 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:58.495076    2570 scope.go:117] "RemoveContainer" containerID="9afd5a34f7b4749afc48a8659e8c9eaaf2f74cda13ce8596f3f472c2ae56f32e"
Sep 17 14:26:58.817459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:26:58.817371    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:27:02.816966 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:02.816928    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" podUID="4246008e-ef59-49c7-b98e-382c40f70986" containerName="main" containerID="cri-o://15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582" gracePeriod=45
Sep 17 14:27:03.076175 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.076104    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:27:03.185867 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.185831    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186055 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.185925    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186055 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.185957    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186055 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.185984    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186055 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186018    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gj8x\" (UniqueName: \"kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186273 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186136    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir\") pod \"4246008e-ef59-49c7-b98e-382c40f70986\" (UID: \"4246008e-ef59-49c7-b98e-382c40f70986\") "
Sep 17 14:27:03.186273 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186152    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache" (OuterVolumeSpecName: "model-cache") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:27:03.186273 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186249    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home" (OuterVolumeSpecName: "home") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:27:03.186426 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186318    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:27:03.186540 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186523    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.186619 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186543    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.186619 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.186552    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.188404 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.188380    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm" (OuterVolumeSpecName: "dshm") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:27:03.188529 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.188426    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:27:03.188529 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.188496    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x" (OuterVolumeSpecName: "kube-api-access-9gj8x") pod "4246008e-ef59-49c7-b98e-382c40f70986" (UID: "4246008e-ef59-49c7-b98e-382c40f70986"). InnerVolumeSpecName "kube-api-access-9gj8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:27:03.287380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.287349    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4246008e-ef59-49c7-b98e-382c40f70986-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.287380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.287376    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4246008e-ef59-49c7-b98e-382c40f70986-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.287380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.287387    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gj8x\" (UniqueName: \"kubernetes.io/projected/4246008e-ef59-49c7-b98e-382c40f70986-kube-api-access-9gj8x\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:03.307215 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.307181    2570 generic.go:356] "Generic (PLEG): container finished" podID="4246008e-ef59-49c7-b98e-382c40f70986" containerID="15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582" exitCode=0
Sep 17 14:27:03.307387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.307234    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"
Sep 17 14:27:03.307387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.307258    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" event={"ID":"4246008e-ef59-49c7-b98e-382c40f70986","Type":"ContainerDied","Data":"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"}
Sep 17 14:27:03.307387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.307302    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj" event={"ID":"4246008e-ef59-49c7-b98e-382c40f70986","Type":"ContainerDied","Data":"f4a1379aab8a510cede34764363deec30cd67def9942679d2e3e3c929f240191"}
Sep 17 14:27:03.307387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.307321    2570 scope.go:117] "RemoveContainer" containerID="15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"
Sep 17 14:27:03.317176 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.317153    2570 scope.go:117] "RemoveContainer" containerID="15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"
Sep 17 14:27:03.317466 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:03.317447    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582\": container with ID starting with 15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582 not found: ID does not exist" containerID="15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"
Sep 17 14:27:03.317518 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.317477    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582"} err="failed to get container status \"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582\": rpc error: code = NotFound desc = could not find container \"15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582\": container with ID starting with 15b47bee493579aa6f1665c1e0fae3e50d685e42fca9df5de9c1551b0f8be582 not found: ID does not exist"
Sep 17 14:27:03.330013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.329938    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:27:03.335188 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:03.335155    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/auth-test-model-b88b90-kserve-5784ddb667-szjvj"]
Sep 17 14:27:04.181545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:04.181508    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4246008e-ef59-49c7-b98e-382c40f70986" path="/var/lib/kubelet/pods/4246008e-ef59-49c7-b98e-382c40f70986/volumes"
Sep 17 14:27:23.435491 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:23.435456    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:27:37.817554 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:37.817515    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:27:47.514782 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:47.514726    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:27:49.968884 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.968848    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7777-673f2e"]
Sep 17 14:27:49.969372 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969357    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4246008e-ef59-49c7-b98e-382c40f70986" containerName="main"
Sep 17 14:27:49.969422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969377    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4246008e-ef59-49c7-b98e-382c40f70986" containerName="main"
Sep 17 14:27:49.969422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969396    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" containerName="main"
Sep 17 14:27:49.969422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969404    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" containerName="main"
Sep 17 14:27:49.969422 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969416    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c922a27e-de29-47f2-8e6e-47c1347ed55b" containerName="payload-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969425    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c922a27e-de29-47f2-8e6e-47c1347ed55b" containerName="payload-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969439    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da07655d-388b-410d-b755-9b8d755d834e" containerName="main"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969446    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="da07655d-388b-410d-b755-9b8d755d834e" containerName="main"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969460    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969467    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969475    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7059ca5d-dd99-454a-9216-86af0c7cbd96" containerName="payload-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969483    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7059ca5d-dd99-454a-9216-86af0c7cbd96" containerName="payload-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969493    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969500    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969514    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbc1905-f574-4b68-8d7a-4f40056edebb" containerName="payload-pre-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969522    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbc1905-f574-4b68-8d7a-4f40056edebb" containerName="payload-pre-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969533    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c259d2-ea17-4a75-afa8-8db691845ecc" containerName="payload-processing"
Sep 17 14:27:49.969545 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969541    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c259d2-ea17-4a75-afa8-8db691845ecc" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969552    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec571499-5c08-4034-a391-164356db00c1" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969560    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec571499-5c08-4034-a391-164356db00c1" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969570    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ff4d3c-4164-4965-975e-2bcb0457f594" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969578    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ff4d3c-4164-4965-975e-2bcb0457f594" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969588    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969596    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969608    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969616    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969697    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4246008e-ef59-49c7-b98e-382c40f70986" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969706    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6afa62d0-95db-4615-a6bf-ab2bc5b5eee7" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969714    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c922a27e-de29-47f2-8e6e-47c1347ed55b" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969723    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7059ca5d-dd99-454a-9216-86af0c7cbd96" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969747    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c259d2-ea17-4a75-afa8-8db691845ecc" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969756    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969764    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ff4d3c-4164-4965-975e-2bcb0457f594" containerName="payload-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969772    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="da07655d-388b-410d-b755-9b8d755d834e" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969779    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbc1905-f574-4b68-8d7a-4f40056edebb" containerName="payload-pre-processing"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969786    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="50564ad9-11a8-452c-bf84-1746292bd529" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969799    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969806    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec571499-5c08-4034-a391-164356db00c1" containerName="main"
Sep 17 14:27:49.969987 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.969814    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="982631a4-7721-44f1-9032-377754e77c64" containerName="istio-proxy"
Sep 17 14:27:49.972930 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.972913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:49.975914 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.975894    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-f8z2f\""
Sep 17 14:27:49.981199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:49.981179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7777-673f2e"]
Sep 17 14:27:50.023853 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:50.023810    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqs9m\" (UniqueName: \"kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m\") pod \"test-curl-7777-673f2e\" (UID: \"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440\") " pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:50.125347 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:50.125308    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqs9m\" (UniqueName: \"kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m\") pod \"test-curl-7777-673f2e\" (UID: \"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440\") " pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:50.134530 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:50.134506    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqs9m\" (UniqueName: \"kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m\") pod \"test-curl-7777-673f2e\" (UID: \"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440\") " pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:50.283009 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:50.282922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:50.554908 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:50.554885    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7777-673f2e"]
Sep 17 14:27:50.557553 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:27:50.557522    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75e3fc2a_a7c5_4490_9ba4_0b4867b5a440.slice/crio-71b661b182852d33cb4791fbff3794c242b5502b0a61098b64041e0d3eacfee9 WatchSource:0}: Error finding container 71b661b182852d33cb4791fbff3794c242b5502b0a61098b64041e0d3eacfee9: Status 404 returned error can't find the container with id 71b661b182852d33cb4791fbff3794c242b5502b0a61098b64041e0d3eacfee9
Sep 17 14:27:51.545785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:51.545748    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7777-673f2e" event={"ID":"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440","Type":"ContainerStarted","Data":"eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88"}
Sep 17 14:27:51.545785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:51.545789    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7777-673f2e" event={"ID":"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440","Type":"ContainerStarted","Data":"71b661b182852d33cb4791fbff3794c242b5502b0a61098b64041e0d3eacfee9"}
Sep 17 14:27:51.569754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:51.569686    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7777-673f2e" podStartSLOduration=2.569670418 podStartE2EDuration="2.569670418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:27:51.566752513 +0000 UTC m=+1913.803629981" watchObservedRunningTime="2026-09-17 14:27:51.569670418 +0000 UTC m=+1913.806547896"
Sep 17 14:27:52.324828 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.324741    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7777-673f2e"]
Sep 17 14:27:52.331649 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.331613    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7777-673f2e"]
Sep 17 14:27:52.457269 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.457232    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7777-35c9cd"]
Sep 17 14:27:52.457629 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.457617    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" containerName="test-curl-7777-673f2e"
Sep 17 14:27:52.457674 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.457632    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" containerName="test-curl-7777-673f2e"
Sep 17 14:27:52.457718 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.457709    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" containerName="test-curl-7777-673f2e"
Sep 17 14:27:52.460690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.460675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:52.474442 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.474410    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7777-35c9cd"]
Sep 17 14:27:52.548923 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.548883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbdhm\" (UniqueName: \"kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm\") pod \"test-curl-7777-35c9cd\" (UID: \"4cec24a5-1764-4319-a311-08b518b6614b\") " pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:52.551682 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:52.551654    2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7777-673f2e" container="test-curl-7777-673f2e"
Sep 17 14:27:52.649864 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.649777    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbdhm\" (UniqueName: \"kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm\") pod \"test-curl-7777-35c9cd\" (UID: \"4cec24a5-1764-4319-a311-08b518b6614b\") " pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:52.660300 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.660266    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbdhm\" (UniqueName: \"kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm\") pod \"test-curl-7777-35c9cd\" (UID: \"4cec24a5-1764-4319-a311-08b518b6614b\") " pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:52.769848 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:52.769810    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:53.046479 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:53.046447    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7777-35c9cd"]
Sep 17 14:27:53.048028 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:27:53.047982    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cec24a5_1764_4319_a311_08b518b6614b.slice/crio-ee780587a6813343e67af527f29b3ea0d0f4419ec911b92a0c3dff5117f2a3fb WatchSource:0}: Error finding container ee780587a6813343e67af527f29b3ea0d0f4419ec911b92a0c3dff5117f2a3fb: Status 404 returned error can't find the container with id ee780587a6813343e67af527f29b3ea0d0f4419ec911b92a0c3dff5117f2a3fb
Sep 17 14:27:53.559076 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:53.559032    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7777-35c9cd" event={"ID":"4cec24a5-1764-4319-a311-08b518b6614b","Type":"ContainerStarted","Data":"a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803"}
Sep 17 14:27:53.559561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:53.559084    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7777-35c9cd" event={"ID":"4cec24a5-1764-4319-a311-08b518b6614b","Type":"ContainerStarted","Data":"ee780587a6813343e67af527f29b3ea0d0f4419ec911b92a0c3dff5117f2a3fb"}
Sep 17 14:27:53.559561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:53.559170    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7777-673f2e" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" containerName="test-curl-7777-673f2e" containerID="cri-o://eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88" gracePeriod=2
Sep 17 14:27:53.582089 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:53.582051    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7777-673f2e\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:54.245058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:54.244954    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7777-35c9cd" podStartSLOduration=2.244936502 podStartE2EDuration="2.244936502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:27:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:27:53.579475154 +0000 UTC m=+1915.816352632" watchObservedRunningTime="2026-09-17 14:27:54.244936502 +0000 UTC m=+1916.481813979"
Sep 17 14:27:54.246511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:54.246492    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7777-35c9cd"]
Sep 17 14:27:54.249714 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:54.249688    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7777-35c9cd"]
Sep 17 14:27:54.564057 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:54.563969    2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7777-35c9cd" container="test-curl-7777-35c9cd"
Sep 17 14:27:55.569166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:55.569133    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7777-35c9cd" podUID="4cec24a5-1764-4319-a311-08b518b6614b" containerName="test-curl-7777-35c9cd" containerID="cri-o://a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803" gracePeriod=2
Sep 17 14:27:55.571713 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:55.571673    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7777-35c9cd\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4cec24a5-1764-4319-a311-08b518b6614b" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:55.726480 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:55.726456    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:55.729066 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:55.729037    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7777-35c9cd\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="4cec24a5-1764-4319-a311-08b518b6614b" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:55.879687 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:55.879595    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqs9m\" (UniqueName: \"kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m\") pod \"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440\" (UID: \"75e3fc2a-a7c5-4490-9ba4-0b4867b5a440\") "
Sep 17 14:27:55.881951 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:55.881921    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m" (OuterVolumeSpecName: "kube-api-access-tqs9m") pod "75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" (UID: "75e3fc2a-a7c5-4490-9ba4-0b4867b5a440"). InnerVolumeSpecName "kube-api-access-tqs9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:27:55.980775 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:55.980714    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqs9m\" (UniqueName: \"kubernetes.io/projected/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440-kube-api-access-tqs9m\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:56.187625 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.187588    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" path="/var/lib/kubelet/pods/75e3fc2a-a7c5-4490-9ba4-0b4867b5a440/volumes"
Sep 17 14:27:56.575126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.575039    2570 generic.go:356] "Generic (PLEG): container finished" podID="75e3fc2a-a7c5-4490-9ba4-0b4867b5a440" containerID="eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88" exitCode=137
Sep 17 14:27:56.575126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.575099    2570 scope.go:117] "RemoveContainer" containerID="eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88"
Sep 17 14:27:56.575126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.575107    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-673f2e"
Sep 17 14:27:56.583862 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.583837    2570 scope.go:117] "RemoveContainer" containerID="eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88"
Sep 17 14:27:56.584129 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:56.584107    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88\": container with ID starting with eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88 not found: ID does not exist" containerID="eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88"
Sep 17 14:27:56.584190 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:56.584140    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88"} err="failed to get container status \"eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88\": rpc error: code = NotFound desc = could not find container \"eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88\": container with ID starting with eae6b3a8a2a59febc541578d31608a9b91782512ce546f991e4345c9591fcc88 not found: ID does not exist"
Sep 17 14:27:57.758768 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:57.758744    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:57.899983 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:57.899898    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbdhm\" (UniqueName: \"kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm\") pod \"4cec24a5-1764-4319-a311-08b518b6614b\" (UID: \"4cec24a5-1764-4319-a311-08b518b6614b\") "
Sep 17 14:27:57.902104 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:57.902077    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm" (OuterVolumeSpecName: "kube-api-access-kbdhm") pod "4cec24a5-1764-4319-a311-08b518b6614b" (UID: "4cec24a5-1764-4319-a311-08b518b6614b"). InnerVolumeSpecName "kube-api-access-kbdhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:27:58.001653 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.001592    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbdhm\" (UniqueName: \"kubernetes.io/projected/4cec24a5-1764-4319-a311-08b518b6614b-kube-api-access-kbdhm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:27:58.181850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.181822    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cec24a5-1764-4319-a311-08b518b6614b" path="/var/lib/kubelet/pods/4cec24a5-1764-4319-a311-08b518b6614b/volumes"
Sep 17 14:27:58.419411 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.419370    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:27:58.589498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.589461    2570 generic.go:356] "Generic (PLEG): container finished" podID="4cec24a5-1764-4319-a311-08b518b6614b" containerID="a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803" exitCode=137
Sep 17 14:27:58.589498 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.589504    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7777-35c9cd"
Sep 17 14:27:58.589711 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.589548    2570 scope.go:117] "RemoveContainer" containerID="a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803"
Sep 17 14:27:58.597587 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.597568    2570 scope.go:117] "RemoveContainer" containerID="a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803"
Sep 17 14:27:58.597867 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:27:58.597849    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803\": container with ID starting with a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803 not found: ID does not exist" containerID="a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803"
Sep 17 14:27:58.597931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:27:58.597879    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803"} err="failed to get container status \"a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803\": rpc error: code = NotFound desc = could not find container \"a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803\": container with ID starting with a29490d41944830e46655130be5374f3f47c7ad3a76eb8921da0cb90dfdea803 not found: ID does not exist"
Sep 17 14:28:07.725995 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:07.725946    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:28:11.294556 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.294519    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:28:11.295136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.295119    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cec24a5-1764-4319-a311-08b518b6614b" containerName="test-curl-7777-35c9cd"
Sep 17 14:28:11.295188 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.295139    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cec24a5-1764-4319-a311-08b518b6614b" containerName="test-curl-7777-35c9cd"
Sep 17 14:28:11.295253 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.295241    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cec24a5-1764-4319-a311-08b518b6614b" containerName="test-curl-7777-35c9cd"
Sep 17 14:28:11.298651 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.298629    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.308482 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.308458    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"openshift-service-ca.crt\""
Sep 17 14:28:11.308643 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.308593    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"kube-root-ca.crt\""
Sep 17 14:28:11.308643 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.308605    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"default-dockercfg-l84qx\""
Sep 17 14:28:11.308643 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.308624    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b88b90\"/\"rate-test-model-b88b90-kserve-self-signed-certs\""
Sep 17 14:28:11.329022 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.328989    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:28:11.423809 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.423779    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.423994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.423829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.423994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.423888    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.423994 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.423926    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.424137 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.424057    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snxsb\" (UniqueName: \"kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.424137 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.424091    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525455 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525412    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snxsb\" (UniqueName: \"kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525455 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525527    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525754 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525580    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.525976 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.526034 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.525974    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.526091 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.526037    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.528291 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.528237    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.529016 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.528991    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.535976 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.535952    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snxsb\" (UniqueName: \"kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb\") pod \"rate-test-model-b88b90-kserve-6bc646475f-96kqc\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") " pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.608249 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.608142    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:11.912591 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:11.912515    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:28:11.914853 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:28:11.914826    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e7756b9_13bf_4ede_b46e_80d60ea3d061.slice/crio-6e1de11852f5e6b289cfecd36e4f85966f0c2ade74c45cb8dc0b7faf4436fd14 WatchSource:0}: Error finding container 6e1de11852f5e6b289cfecd36e4f85966f0c2ade74c45cb8dc0b7faf4436fd14: Status 404 returned error can't find the container with id 6e1de11852f5e6b289cfecd36e4f85966f0c2ade74c45cb8dc0b7faf4436fd14
Sep 17 14:28:12.661057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:12.661016    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" event={"ID":"0e7756b9-13bf-4ede-b46e-80d60ea3d061","Type":"ContainerStarted","Data":"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"}
Sep 17 14:28:12.661057 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:12.661062    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" event={"ID":"0e7756b9-13bf-4ede-b46e-80d60ea3d061","Type":"ContainerStarted","Data":"6e1de11852f5e6b289cfecd36e4f85966f0c2ade74c45cb8dc0b7faf4436fd14"}
Sep 17 14:28:12.696812 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:12.696766    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" podStartSLOduration=1.696723497 podStartE2EDuration="1.696723497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:28:12.696557287 +0000 UTC m=+1934.933434764" watchObservedRunningTime="2026-09-17 14:28:12.696723497 +0000 UTC m=+1934.933601025"
Sep 17 14:28:13.533381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:13.533346    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:28:21.608460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:21.608412    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:21.608908 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:21.608482    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:21.620784 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:21.620757    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:21.722149 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:21.722117    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:28:42.578652 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:42.578612    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:28:55.623886 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:28:55.623807    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:29:04.926912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:04.926880    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:29:17.713409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:17.713371    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:29:21.800063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:21.800031    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:29:26.317619 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:26.317581    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:29:36.814216 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:36.814179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:29:36.818848 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:36.818822    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" podUID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" containerName="main" containerID="cri-o://daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53" gracePeriod=45
Sep 17 14:29:37.069388 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.069331    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:29:37.084140 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.084111    2570 generic.go:356] "Generic (PLEG): container finished" podID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" containerID="daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53" exitCode=0
Sep 17 14:29:37.084261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.084149    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"
Sep 17 14:29:37.084261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.084179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" event={"ID":"0e7756b9-13bf-4ede-b46e-80d60ea3d061","Type":"ContainerDied","Data":"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"}
Sep 17 14:29:37.084261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.084214    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc" event={"ID":"0e7756b9-13bf-4ede-b46e-80d60ea3d061","Type":"ContainerDied","Data":"6e1de11852f5e6b289cfecd36e4f85966f0c2ade74c45cb8dc0b7faf4436fd14"}
Sep 17 14:29:37.084261 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.084236    2570 scope.go:117] "RemoveContainer" containerID="daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"
Sep 17 14:29:37.093295 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.093279    2570 scope.go:117] "RemoveContainer" containerID="daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"
Sep 17 14:29:37.093581 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:29:37.093561    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53\": container with ID starting with daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53 not found: ID does not exist" containerID="daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"
Sep 17 14:29:37.093665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.093589    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53"} err="failed to get container status \"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53\": rpc error: code = NotFound desc = could not find container \"daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53\": container with ID starting with daac280cd1500389977ccdcb1312775717db98fddeade3a54eb89d19ad4f5d53 not found: ID does not exist"
Sep 17 14:29:37.189747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189712    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.189912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189808    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.189912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189827    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snxsb\" (UniqueName: \"kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.189912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189881    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.189912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189897    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.190133 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189941    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home\") pod \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\" (UID: \"0e7756b9-13bf-4ede-b46e-80d60ea3d061\") "
Sep 17 14:29:37.190133 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.189955    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:37.190242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.190127    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:37.190242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.190141    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.190311 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.190261    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home" (OuterVolumeSpecName: "home") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:37.192069 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.192050    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm" (OuterVolumeSpecName: "dshm") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:29:37.192158 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.192131    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:29:37.192206 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.192174    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb" (OuterVolumeSpecName: "kube-api-access-snxsb") pod "0e7756b9-13bf-4ede-b46e-80d60ea3d061" (UID: "0e7756b9-13bf-4ede-b46e-80d60ea3d061"). InnerVolumeSpecName "kube-api-access-snxsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:29:37.291521 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.291473    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.291521 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.291511    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.291521 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.291520    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e7756b9-13bf-4ede-b46e-80d60ea3d061-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.291521 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.291527    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e7756b9-13bf-4ede-b46e-80d60ea3d061-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.291879 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.291536    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snxsb\" (UniqueName: \"kubernetes.io/projected/0e7756b9-13bf-4ede-b46e-80d60ea3d061-kube-api-access-snxsb\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:29:37.406846 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.406821    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:29:37.416806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:37.410931    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b88b90/rate-test-model-b88b90-kserve-6bc646475f-96kqc"]
Sep 17 14:29:38.181151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:29:38.181119    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" path="/var/lib/kubelet/pods/0e7756b9-13bf-4ede-b46e-80d60ea3d061/volumes"
Sep 17 14:30:00.148533 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.148500    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"]
Sep 17 14:30:00.148985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.148905    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" containerName="main"
Sep 17 14:30:00.148985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.148918    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" containerName="main"
Sep 17 14:30:00.148985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.148978    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7756b9-13bf-4ede-b46e-80d60ea3d061" containerName="main"
Sep 17 14:30:00.152390 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.152366    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"]
Sep 17 14:30:00.152506 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.152479    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"
Sep 17 14:30:00.155109 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.155084    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rp9ww\""
Sep 17 14:30:00.155949 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.155933    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"
Sep 17 14:30:00.162392 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.162314    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"]
Sep 17 14:30:00.165673 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.165651    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"]
Sep 17 14:30:00.166371 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.166353    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"
Sep 17 14:30:00.174479 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.174459    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"
Sep 17 14:30:00.447105 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.447086    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"]
Sep 17 14:30:00.449459 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:30:00.449432    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c65d32_5b1f_4f0b_a618_d9abb5a7bacd.slice/crio-c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed WatchSource:0}: Error finding container c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed: Status 404 returned error can't find the container with id c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed
Sep 17 14:30:00.453609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:00.453549    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"]
Sep 17 14:30:00.457012 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:30:00.456987    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db7a284_05a0_48b3_8fef_6e5922ce10e5.slice/crio-4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b WatchSource:0}: Error finding container 4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b: Status 404 returned error can't find the container with id 4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b
Sep 17 14:30:01.212015 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.211976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh" event={"ID":"54c65d32-5b1f-4f0b-a618-d9abb5a7bacd","Type":"ContainerStarted","Data":"adde01262fbb81eb4679e8f5ede7bc440057faa857a471300ffde48921337b03"}
Sep 17 14:30:01.212480 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.212042    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh" event={"ID":"54c65d32-5b1f-4f0b-a618-d9abb5a7bacd","Type":"ContainerStarted","Data":"c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed"}
Sep 17 14:30:01.213617 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.213594    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv" event={"ID":"7db7a284-05a0-48b3-8fef-6e5922ce10e5","Type":"ContainerStarted","Data":"61db8d19387c30d84d7f17a82e5a3f01d05e147d0e544e927d69caee37eff903"}
Sep 17 14:30:01.213744 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.213633    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv" event={"ID":"7db7a284-05a0-48b3-8fef-6e5922ce10e5","Type":"ContainerStarted","Data":"4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b"}
Sep 17 14:30:01.241405 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.241362    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh" podStartSLOduration=1.24134835 podStartE2EDuration="1.24134835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.238842693 +0000 UTC m=+2043.475720173" watchObservedRunningTime="2026-09-17 14:30:01.24134835 +0000 UTC m=+2043.478225826"
Sep 17 14:30:01.275801 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:01.275760    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv" podStartSLOduration=1.275747859 podStartE2EDuration="1.275747859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:01.275222179 +0000 UTC m=+2043.512099656" watchObservedRunningTime="2026-09-17 14:30:01.275747859 +0000 UTC m=+2043.512625327"
Sep 17 14:30:02.220709 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:02.220672    2570 generic.go:356] "Generic (PLEG): container finished" podID="7db7a284-05a0-48b3-8fef-6e5922ce10e5" containerID="61db8d19387c30d84d7f17a82e5a3f01d05e147d0e544e927d69caee37eff903" exitCode=0
Sep 17 14:30:02.221121 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:02.220761    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv" event={"ID":"7db7a284-05a0-48b3-8fef-6e5922ce10e5","Type":"ContainerDied","Data":"61db8d19387c30d84d7f17a82e5a3f01d05e147d0e544e927d69caee37eff903"}
Sep 17 14:30:02.222199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:02.222177    2570 generic.go:356] "Generic (PLEG): container finished" podID="54c65d32-5b1f-4f0b-a618-d9abb5a7bacd" containerID="adde01262fbb81eb4679e8f5ede7bc440057faa857a471300ffde48921337b03" exitCode=0
Sep 17 14:30:02.222268 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:02.222223    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh" event={"ID":"54c65d32-5b1f-4f0b-a618-d9abb5a7bacd","Type":"ContainerDied","Data":"adde01262fbb81eb4679e8f5ede7bc440057faa857a471300ffde48921337b03"}
Sep 17 14:30:03.381380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:03.381352    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"
Sep 17 14:30:03.384162 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:03.384144    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"
Sep 17 14:30:04.233126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.233103    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"
Sep 17 14:30:04.233126 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.233104    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv" event={"ID":"7db7a284-05a0-48b3-8fef-6e5922ce10e5","Type":"ContainerDied","Data":"4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b"}
Sep 17 14:30:04.233361 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.233154    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f1d0394a2f04f6c92648b0aa9f03ad87df10878e13ea7725ffe87efc40e253b"
Sep 17 14:30:04.234530 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.234509    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh" event={"ID":"54c65d32-5b1f-4f0b-a618-d9abb5a7bacd","Type":"ContainerDied","Data":"c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed"}
Sep 17 14:30:04.234659 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.234542    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c13e153174224981eb7012b3ff4b3ab15248ccc0487c7dda6f2792ce9ed2cfed"
Sep 17 14:30:04.234659 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.234550    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"
Sep 17 14:30:04.864650 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.864607    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:30:04.865044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.864991    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7db7a284-05a0-48b3-8fef-6e5922ce10e5" containerName="cleanup"
Sep 17 14:30:04.865044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.865003    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7db7a284-05a0-48b3-8fef-6e5922ce10e5" containerName="cleanup"
Sep 17 14:30:04.865044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.865019    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c65d32-5b1f-4f0b-a618-d9abb5a7bacd" containerName="cleanup"
Sep 17 14:30:04.865044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.865024    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c65d32-5b1f-4f0b-a618-d9abb5a7bacd" containerName="cleanup"
Sep 17 14:30:04.865176 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.865088    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7db7a284-05a0-48b3-8fef-6e5922ce10e5" containerName="cleanup"
Sep 17 14:30:04.865176 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.865096    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c65d32-5b1f-4f0b-a618-d9abb5a7bacd" containerName="cleanup"
Sep 17 14:30:04.868030 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.868013    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:04.871152 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.871129    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-166f16-serving-cert\""
Sep 17 14:30:04.880611 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.880590    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:30:04.933380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.933345    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:04.933380 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.933379    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:04.933621 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.933402    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:04.933621 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:04.933510    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp29l\" (UniqueName: \"kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.034037 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.034005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.034037 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.034037    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.034285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.034058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.034285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.034100    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp29l\" (UniqueName: \"kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.034808 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.034789    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.036580 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.036556    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.036681 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.036588    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.042883 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.042861    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp29l\" (UniqueName: \"kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l\") pod \"maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.179010 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.178982    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:05.456427 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.456394    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:30:05.458038 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:30:05.458011    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68b7d34_2287_4ee9_9113_99527ae89b46.slice/crio-a29cc5ab5dfb33263b1ad7af115c33c1af711284158cc2d64a458fd51108b290 WatchSource:0}: Error finding container a29cc5ab5dfb33263b1ad7af115c33c1af711284158cc2d64a458fd51108b290: Status 404 returned error can't find the container with id a29cc5ab5dfb33263b1ad7af115c33c1af711284158cc2d64a458fd51108b290
Sep 17 14:30:05.459430 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.459412    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:30:05.619359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.619319    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:30:05.623985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.623962    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.627434 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.627414    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-166f16\""
Sep 17 14:30:05.633579 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.633556    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:30:05.641296 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.641272    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnq8w\" (UniqueName: \"kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.641407 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.641328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.742848 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.742724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.743106 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.743084    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnq8w\" (UniqueName: \"kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.743497 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.743477    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.752565 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.752539    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnq8w\" (UniqueName: \"kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w\") pod \"payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:05.935328 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:05.935287    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:06.206301 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:06.206275    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:30:06.208428 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:30:06.208401    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9654db8_a35d_44fd_8da9_38728ebdc944.slice/crio-7f84d6c509d44e2e49b6a91c1f70845e0f679c5142ab6ca9be40f8f8dffc81fb WatchSource:0}: Error finding container 7f84d6c509d44e2e49b6a91c1f70845e0f679c5142ab6ca9be40f8f8dffc81fb: Status 404 returned error can't find the container with id 7f84d6c509d44e2e49b6a91c1f70845e0f679c5142ab6ca9be40f8f8dffc81fb
Sep 17 14:30:06.255634 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:06.255605    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" event={"ID":"a9654db8-a35d-44fd-8da9-38728ebdc944","Type":"ContainerStarted","Data":"7f84d6c509d44e2e49b6a91c1f70845e0f679c5142ab6ca9be40f8f8dffc81fb"}
Sep 17 14:30:06.257162 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:06.257133    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" event={"ID":"d68b7d34-2287-4ee9-9113-99527ae89b46","Type":"ContainerStarted","Data":"a29cc5ab5dfb33263b1ad7af115c33c1af711284158cc2d64a458fd51108b290"}
Sep 17 14:30:07.265281 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.265245    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" event={"ID":"a9654db8-a35d-44fd-8da9-38728ebdc944","Type":"ContainerStarted","Data":"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"}
Sep 17 14:30:07.265761 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.265352    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:07.267349 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.267318    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" event={"ID":"d68b7d34-2287-4ee9-9113-99527ae89b46","Type":"ContainerStarted","Data":"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"}
Sep 17 14:30:07.267460 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.267449    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:07.285786 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.285726    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" podStartSLOduration=2.285708475 podStartE2EDuration="2.285708475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:07.282516217 +0000 UTC m=+2049.519393694" watchObservedRunningTime="2026-09-17 14:30:07.285708475 +0000 UTC m=+2049.522585951"
Sep 17 14:30:07.304341 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:07.304302    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" podStartSLOduration=2.502399604 podStartE2EDuration="3.304287547s" totalImagesPullingTime="801.887943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:30:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:30:07.302458635 +0000 UTC m=+2049.539336115" watchObservedRunningTime="2026-09-17 14:30:07.304287547 +0000 UTC m=+2049.541165041"
Sep 17 14:30:09.212465 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:09.212424    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:30:13.278114 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:13.278087    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:30:18.274941 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:18.274911    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:30:53.012898 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:30:53.012859    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:31:06.931234 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:06.931201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:31:21.407456 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.407416    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:31:21.412464 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.412439    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.415148 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.415124    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-cd2006-serving-cert\""
Sep 17 14:31:21.421103 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.421080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:31:21.492642 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.492603    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.492838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.492666    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6vs8\" (UniqueName: \"kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.492838 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.492803    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.492950 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.492842    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.594103 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.594070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.594301 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.594110    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.594301 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.594161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.594301 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.594194    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6vs8\" (UniqueName: \"kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.594985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.594962    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.596695 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.596668    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.596887 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.596867    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.603642 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.603619    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6vs8\" (UniqueName: \"kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8\") pod \"maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:21.725792 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:21.725710    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:22.004424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.004399    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:31:22.007565 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:31:22.007524    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2de614d_444a_4728_821b_860e4dd755be.slice/crio-62f1a8f7d0a208112292887412c4830b8c9d2149e3b31759ee3e42177cb09977 WatchSource:0}: Error finding container 62f1a8f7d0a208112292887412c4830b8c9d2149e3b31759ee3e42177cb09977: Status 404 returned error can't find the container with id 62f1a8f7d0a208112292887412c4830b8c9d2149e3b31759ee3e42177cb09977
Sep 17 14:31:22.102911 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.102881    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:31:22.110716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.109325    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.113549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.113523    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:31:22.114322 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.114305    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-cd2006-dockercfg-p42l7\""
Sep 17 14:31:22.114964 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.114950    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-cd2006\""
Sep 17 14:31:22.117360 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.117323    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.120082 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.120055    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:31:22.128410 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.128386    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:31:22.199367 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.199338    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.199541 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.199370    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fnlv\" (UniqueName: \"kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.199541 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.199413    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.199541 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.199472    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22ssq\" (UniqueName: \"kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.300917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.300827    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.300917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.300874    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22ssq\" (UniqueName: \"kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.301144 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.300983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.301144 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.301007    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fnlv\" (UniqueName: \"kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.301547 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.301527    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.301678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.301660    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.313110 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.313090    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fnlv\" (UniqueName: \"kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv\") pod \"payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.313219 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.313187    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22ssq\" (UniqueName: \"kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq\") pod \"payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.422921 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.422889    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:22.431487 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.431469    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:22.643478 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.643447    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" event={"ID":"b2de614d-444a-4728-821b-860e4dd755be","Type":"ContainerStarted","Data":"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"}
Sep 17 14:31:22.643627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.643498    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" event={"ID":"b2de614d-444a-4728-821b-860e4dd755be","Type":"ContainerStarted","Data":"62f1a8f7d0a208112292887412c4830b8c9d2149e3b31759ee3e42177cb09977"}
Sep 17 14:31:22.643627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.643553    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:22.674971 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.674915    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" podStartSLOduration=1.169579595 podStartE2EDuration="1.674898022s" totalImagesPullingTime="505.318427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:31:22.67362949 +0000 UTC m=+2124.910506967" watchObservedRunningTime="2026-09-17 14:31:22.674898022 +0000 UTC m=+2124.911775530"
Sep 17 14:31:22.705884 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.705769    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:31:22.708776 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:31:22.708720    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d6a177_3f10_469f_a763_ebef283ecd41.slice/crio-b82f52a8e95b58dfd0daa8dd226c2fdeb3711d4ad72db656a86e643cc7ca0744 WatchSource:0}: Error finding container b82f52a8e95b58dfd0daa8dd226c2fdeb3711d4ad72db656a86e643cc7ca0744: Status 404 returned error can't find the container with id b82f52a8e95b58dfd0daa8dd226c2fdeb3711d4ad72db656a86e643cc7ca0744
Sep 17 14:31:22.724103 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:22.724082    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:31:22.726985 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:31:22.726957    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e160003_d84a_4f71_8648_6880443969e8.slice/crio-4e48750b0907f10bcb3571efc8ace2f95516a025307fa9c6f470c635af1d897d WatchSource:0}: Error finding container 4e48750b0907f10bcb3571efc8ace2f95516a025307fa9c6f470c635af1d897d: Status 404 returned error can't find the container with id 4e48750b0907f10bcb3571efc8ace2f95516a025307fa9c6f470c635af1d897d
Sep 17 14:31:23.650669 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.650625    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" event={"ID":"63d6a177-3f10-469f-a763-ebef283ecd41","Type":"ContainerStarted","Data":"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"}
Sep 17 14:31:23.651207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.650702    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" event={"ID":"63d6a177-3f10-469f-a763-ebef283ecd41","Type":"ContainerStarted","Data":"b82f52a8e95b58dfd0daa8dd226c2fdeb3711d4ad72db656a86e643cc7ca0744"}
Sep 17 14:31:23.651207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.650757    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:23.652875 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.652843    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" event={"ID":"8e160003-d84a-4f71-8648-6880443969e8","Type":"ContainerStarted","Data":"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"}
Sep 17 14:31:23.653030 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.652911    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" event={"ID":"8e160003-d84a-4f71-8648-6880443969e8","Type":"ContainerStarted","Data":"4e48750b0907f10bcb3571efc8ace2f95516a025307fa9c6f470c635af1d897d"}
Sep 17 14:31:23.653205 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.653176    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:23.672124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.672079    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" podStartSLOduration=1.6720642940000001 podStartE2EDuration="1.672064294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:31:23.66978412 +0000 UTC m=+2125.906661600" watchObservedRunningTime="2026-09-17 14:31:23.672064294 +0000 UTC m=+2125.908941771"
Sep 17 14:31:23.690453 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:23.690406    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" podStartSLOduration=1.6903921560000001 podStartE2EDuration="1.690392156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:31:23.689500251 +0000 UTC m=+2125.926377729" watchObservedRunningTime="2026-09-17 14:31:23.690392156 +0000 UTC m=+2125.927269633"
Sep 17 14:31:26.722159 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:26.722126    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:31:28.658595 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:28.658567    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:31:34.660256 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:34.660224    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:31:34.660666 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:34.660338    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:31:59.803529 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.803440    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:31:59.804135 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.803649    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" podUID="b2de614d-444a-4728-821b-860e4dd755be" containerName="maas-api" containerID="cri-o://134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127" gracePeriod=30
Sep 17 14:31:59.858018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.857979    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:31:59.858255 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.858206    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" podUID="8e160003-d84a-4f71-8648-6880443969e8" containerName="payload-processing" containerID="cri-o://3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef" gracePeriod=30
Sep 17 14:31:59.868013 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.867983    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:31:59.868216 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:31:59.868194    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" podUID="63d6a177-3f10-469f-a763-ebef283ecd41" containerName="payload-pre-processing" containerID="cri-o://cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55" gracePeriod=30
Sep 17 14:32:00.133224 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.133192    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:32:00.167413 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.167389    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:32:00.173096 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.173078    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:32:00.249500 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249461    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls\") pod \"b2de614d-444a-4728-821b-860e4dd755be\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") "
Sep 17 14:32:00.249678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249553    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fnlv\" (UniqueName: \"kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv\") pod \"8e160003-d84a-4f71-8648-6880443969e8\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") "
Sep 17 14:32:00.249678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249574    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6vs8\" (UniqueName: \"kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8\") pod \"b2de614d-444a-4728-821b-860e4dd755be\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") "
Sep 17 14:32:00.249678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249609    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs\") pod \"b2de614d-444a-4728-821b-860e4dd755be\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") "
Sep 17 14:32:00.249678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249639    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca\") pod \"b2de614d-444a-4728-821b-860e4dd755be\" (UID: \"b2de614d-444a-4728-821b-860e4dd755be\") "
Sep 17 14:32:00.249678 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249662    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume\") pod \"63d6a177-3f10-469f-a763-ebef283ecd41\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") "
Sep 17 14:32:00.249974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249705    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22ssq\" (UniqueName: \"kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq\") pod \"63d6a177-3f10-469f-a763-ebef283ecd41\" (UID: \"63d6a177-3f10-469f-a763-ebef283ecd41\") "
Sep 17 14:32:00.249974 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.249785    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume\") pod \"8e160003-d84a-4f71-8648-6880443969e8\" (UID: \"8e160003-d84a-4f71-8648-6880443969e8\") "
Sep 17 14:32:00.250078 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.250036    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2de614d-444a-4728-821b-860e4dd755be" (UID: "b2de614d-444a-4728-821b-860e4dd755be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:00.250524 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.250220    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2de614d-444a-4728-821b-860e4dd755be-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.250864 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.250840    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8e160003-d84a-4f71-8648-6880443969e8" (UID: "8e160003-d84a-4f71-8648-6880443969e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:00.252181 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.252152    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63d6a177-3f10-469f-a763-ebef283ecd41" (UID: "63d6a177-3f10-469f-a763-ebef283ecd41"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:32:00.252677 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.252647    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv" (OuterVolumeSpecName: "kube-api-access-5fnlv") pod "8e160003-d84a-4f71-8648-6880443969e8" (UID: "8e160003-d84a-4f71-8648-6880443969e8"). InnerVolumeSpecName "kube-api-access-5fnlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:00.252827 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.252806    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b2de614d-444a-4728-821b-860e4dd755be" (UID: "b2de614d-444a-4728-821b-860e4dd755be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:32:00.253072 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.253051    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq" (OuterVolumeSpecName: "kube-api-access-22ssq") pod "63d6a177-3f10-469f-a763-ebef283ecd41" (UID: "63d6a177-3f10-469f-a763-ebef283ecd41"). InnerVolumeSpecName "kube-api-access-22ssq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:00.253165 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.253150    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8" (OuterVolumeSpecName: "kube-api-access-l6vs8") pod "b2de614d-444a-4728-821b-860e4dd755be" (UID: "b2de614d-444a-4728-821b-860e4dd755be"). InnerVolumeSpecName "kube-api-access-l6vs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:32:00.254207 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.254192    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2de614d-444a-4728-821b-860e4dd755be" (UID: "b2de614d-444a-4728-821b-860e4dd755be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:32:00.351747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351646    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d6a177-3f10-469f-a763-ebef283ecd41-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.351747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351686    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22ssq\" (UniqueName: \"kubernetes.io/projected/63d6a177-3f10-469f-a763-ebef283ecd41-kube-api-access-22ssq\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.351747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351701    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8e160003-d84a-4f71-8648-6880443969e8-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.351747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351715    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.351747 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351750    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fnlv\" (UniqueName: \"kubernetes.io/projected/8e160003-d84a-4f71-8648-6880443969e8-kube-api-access-5fnlv\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.352032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351767    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6vs8\" (UniqueName: \"kubernetes.io/projected/b2de614d-444a-4728-821b-860e4dd755be-kube-api-access-l6vs8\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.352032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.351780    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b2de614d-444a-4728-821b-860e4dd755be-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:32:00.854630 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.854590    2570 generic.go:356] "Generic (PLEG): container finished" podID="8e160003-d84a-4f71-8648-6880443969e8" containerID="3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef" exitCode=0
Sep 17 14:32:00.855182 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.854665    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"
Sep 17 14:32:00.855182 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.854683    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" event={"ID":"8e160003-d84a-4f71-8648-6880443969e8","Type":"ContainerDied","Data":"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"}
Sep 17 14:32:00.855182 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.854806    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp" event={"ID":"8e160003-d84a-4f71-8648-6880443969e8","Type":"ContainerDied","Data":"4e48750b0907f10bcb3571efc8ace2f95516a025307fa9c6f470c635af1d897d"}
Sep 17 14:32:00.855182 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.854841    2570 scope.go:117] "RemoveContainer" containerID="3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"
Sep 17 14:32:00.856357 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.856335    2570 generic.go:356] "Generic (PLEG): container finished" podID="b2de614d-444a-4728-821b-860e4dd755be" containerID="134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127" exitCode=0
Sep 17 14:32:00.856435 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.856405    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"
Sep 17 14:32:00.856435 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.856422    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" event={"ID":"b2de614d-444a-4728-821b-860e4dd755be","Type":"ContainerDied","Data":"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"}
Sep 17 14:32:00.856542 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.856502    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch" event={"ID":"b2de614d-444a-4728-821b-860e4dd755be","Type":"ContainerDied","Data":"62f1a8f7d0a208112292887412c4830b8c9d2149e3b31759ee3e42177cb09977"}
Sep 17 14:32:00.858134 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.858102    2570 generic.go:356] "Generic (PLEG): container finished" podID="63d6a177-3f10-469f-a763-ebef283ecd41" containerID="cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55" exitCode=0
Sep 17 14:32:00.858262 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.858175    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"
Sep 17 14:32:00.858262 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.858189    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" event={"ID":"63d6a177-3f10-469f-a763-ebef283ecd41","Type":"ContainerDied","Data":"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"}
Sep 17 14:32:00.858262 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.858253    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn" event={"ID":"63d6a177-3f10-469f-a763-ebef283ecd41","Type":"ContainerDied","Data":"b82f52a8e95b58dfd0daa8dd226c2fdeb3711d4ad72db656a86e643cc7ca0744"}
Sep 17 14:32:00.866409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.866386    2570 scope.go:117] "RemoveContainer" containerID="3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"
Sep 17 14:32:00.866920 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:32:00.866888    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef\": container with ID starting with 3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef not found: ID does not exist" containerID="3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"
Sep 17 14:32:00.867011 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.866933    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef"} err="failed to get container status \"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef\": rpc error: code = NotFound desc = could not find container \"3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef\": container with ID starting with 3ed35ddaf559e9c6e60058d740fbd17dc3c771f5a19ffa2e435a5527f5542bef not found: ID does not exist"
Sep 17 14:32:00.867011 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.866961    2570 scope.go:117] "RemoveContainer" containerID="134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"
Sep 17 14:32:00.876574 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.876555    2570 scope.go:117] "RemoveContainer" containerID="134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"
Sep 17 14:32:00.876942 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:32:00.876920    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127\": container with ID starting with 134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127 not found: ID does not exist" containerID="134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"
Sep 17 14:32:00.877033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.876948    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127"} err="failed to get container status \"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127\": rpc error: code = NotFound desc = could not find container \"134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127\": container with ID starting with 134ed8f84841e7c01ff868c1ff33aee3dafe220fd4300b4565c3a65e8dc72127 not found: ID does not exist"
Sep 17 14:32:00.877033 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.876965    2570 scope.go:117] "RemoveContainer" containerID="cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"
Sep 17 14:32:00.884954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.884928    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:32:00.886025 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.886011    2570 scope.go:117] "RemoveContainer" containerID="cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"
Sep 17 14:32:00.886367 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:32:00.886345    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55\": container with ID starting with cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55 not found: ID does not exist" containerID="cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"
Sep 17 14:32:00.886455 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.886373    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55"} err="failed to get container status \"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55\": rpc error: code = NotFound desc = could not find container \"cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55\": container with ID starting with cf877939623245dc5691837bc5fa25b6583132d68378b8be794102dbfc1ccc55 not found: ID does not exist"
Sep 17 14:32:00.886816 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.886749    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-cd2006-5bd48785b5-blcdp"]
Sep 17 14:32:00.900097 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.900077    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:32:00.904911 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.904891    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-cd2006-777cdbdd4c-s8kch"]
Sep 17 14:32:00.916192 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.916170    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:32:00.920472 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:00.920448    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-cd2006-697fdf59cd-2qgtn"]
Sep 17 14:32:02.182130 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:02.182093    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63d6a177-3f10-469f-a763-ebef283ecd41" path="/var/lib/kubelet/pods/63d6a177-3f10-469f-a763-ebef283ecd41/volumes"
Sep 17 14:32:02.182522 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:02.182452    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e160003-d84a-4f71-8648-6880443969e8" path="/var/lib/kubelet/pods/8e160003-d84a-4f71-8648-6880443969e8/volumes"
Sep 17 14:32:02.182812 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:02.182798    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2de614d-444a-4728-821b-860e4dd755be" path="/var/lib/kubelet/pods/b2de614d-444a-4728-821b-860e4dd755be/volumes"
Sep 17 14:32:04.522284 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:04.522249    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:32:25.021612 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:25.021575    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:32:26.977296 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:26.977255    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"]
Sep 17 14:32:26.979138 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:26.979112    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-911a82-29827590-szjgh"]
Sep 17 14:32:28.182806 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:28.182766    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c65d32-5b1f-4f0b-a618-d9abb5a7bacd" path="/var/lib/kubelet/pods/54c65d32-5b1f-4f0b-a618-d9abb5a7bacd/volumes"
Sep 17 14:32:33.761917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:32:33.761886    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:33:14.250483 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.250446    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:33:14.251124 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.251093    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" podUID="d68b7d34-2287-4ee9-9113-99527ae89b46" containerName="maas-api" containerID="cri-o://0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56" gracePeriod=30
Sep 17 14:33:14.332916 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.332879    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:33:14.333155 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.333129    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" podUID="a9654db8-a35d-44fd-8da9-38728ebdc944" containerName="payload-pre-processing" containerID="cri-o://9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822" gracePeriod=30
Sep 17 14:33:14.594985 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.594959    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:33:14.635531 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.635489    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:33:14.717825 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.717783    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume\") pod \"a9654db8-a35d-44fd-8da9-38728ebdc944\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") "
Sep 17 14:33:14.717825 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.717829    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnq8w\" (UniqueName: \"kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w\") pod \"a9654db8-a35d-44fd-8da9-38728ebdc944\" (UID: \"a9654db8-a35d-44fd-8da9-38728ebdc944\") "
Sep 17 14:33:14.718052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.717869    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs\") pod \"d68b7d34-2287-4ee9-9113-99527ae89b46\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") "
Sep 17 14:33:14.718052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718020    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls\") pod \"d68b7d34-2287-4ee9-9113-99527ae89b46\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") "
Sep 17 14:33:14.718145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718053    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp29l\" (UniqueName: \"kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l\") pod \"d68b7d34-2287-4ee9-9113-99527ae89b46\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") "
Sep 17 14:33:14.718145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718072    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca\") pod \"d68b7d34-2287-4ee9-9113-99527ae89b46\" (UID: \"d68b7d34-2287-4ee9-9113-99527ae89b46\") "
Sep 17 14:33:14.718393 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718253    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a9654db8-a35d-44fd-8da9-38728ebdc944" (UID: "a9654db8-a35d-44fd-8da9-38728ebdc944"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:14.718536 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718504    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d68b7d34-2287-4ee9-9113-99527ae89b46" (UID: "d68b7d34-2287-4ee9-9113-99527ae89b46"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:14.718803 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.718592    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a9654db8-a35d-44fd-8da9-38728ebdc944-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:14.720349 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.720322    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d68b7d34-2287-4ee9-9113-99527ae89b46" (UID: "d68b7d34-2287-4ee9-9113-99527ae89b46"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:14.720458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.720346    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d68b7d34-2287-4ee9-9113-99527ae89b46" (UID: "d68b7d34-2287-4ee9-9113-99527ae89b46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:14.720458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.720346    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w" (OuterVolumeSpecName: "kube-api-access-lnq8w") pod "a9654db8-a35d-44fd-8da9-38728ebdc944" (UID: "a9654db8-a35d-44fd-8da9-38728ebdc944"). InnerVolumeSpecName "kube-api-access-lnq8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:14.720542 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.720458    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l" (OuterVolumeSpecName: "kube-api-access-jp29l") pod "d68b7d34-2287-4ee9-9113-99527ae89b46" (UID: "d68b7d34-2287-4ee9-9113-99527ae89b46"). InnerVolumeSpecName "kube-api-access-jp29l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:14.820151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.820053    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnq8w\" (UniqueName: \"kubernetes.io/projected/a9654db8-a35d-44fd-8da9-38728ebdc944-kube-api-access-lnq8w\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:14.820151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.820090    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:14.820151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.820099    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d68b7d34-2287-4ee9-9113-99527ae89b46-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:14.820151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.820107    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp29l\" (UniqueName: \"kubernetes.io/projected/d68b7d34-2287-4ee9-9113-99527ae89b46-kube-api-access-jp29l\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:14.820151 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:14.820116    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d68b7d34-2287-4ee9-9113-99527ae89b46-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:15.253741 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.253705    2570 generic.go:356] "Generic (PLEG): container finished" podID="a9654db8-a35d-44fd-8da9-38728ebdc944" containerID="9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822" exitCode=0
Sep 17 14:33:15.254254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.253768    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"
Sep 17 14:33:15.254254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.253795    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" event={"ID":"a9654db8-a35d-44fd-8da9-38728ebdc944","Type":"ContainerDied","Data":"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"}
Sep 17 14:33:15.254254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.253854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq" event={"ID":"a9654db8-a35d-44fd-8da9-38728ebdc944","Type":"ContainerDied","Data":"7f84d6c509d44e2e49b6a91c1f70845e0f679c5142ab6ca9be40f8f8dffc81fb"}
Sep 17 14:33:15.254254 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.253883    2570 scope.go:117] "RemoveContainer" containerID="9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"
Sep 17 14:33:15.255359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.255331    2570 generic.go:356] "Generic (PLEG): container finished" podID="d68b7d34-2287-4ee9-9113-99527ae89b46" containerID="0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56" exitCode=0
Sep 17 14:33:15.255359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.255373    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" event={"ID":"d68b7d34-2287-4ee9-9113-99527ae89b46","Type":"ContainerDied","Data":"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"}
Sep 17 14:33:15.255359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.255388    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"
Sep 17 14:33:15.255669 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.255414    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb" event={"ID":"d68b7d34-2287-4ee9-9113-99527ae89b46","Type":"ContainerDied","Data":"a29cc5ab5dfb33263b1ad7af115c33c1af711284158cc2d64a458fd51108b290"}
Sep 17 14:33:15.263321 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.263299    2570 scope.go:117] "RemoveContainer" containerID="9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"
Sep 17 14:33:15.263634 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:33:15.263584    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822\": container with ID starting with 9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822 not found: ID does not exist" containerID="9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"
Sep 17 14:33:15.263743 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.263663    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822"} err="failed to get container status \"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822\": rpc error: code = NotFound desc = could not find container \"9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822\": container with ID starting with 9bba4f436e12c7563d6ed01ea94749c317e1bfb7dac078b3c7eded38cd122822 not found: ID does not exist"
Sep 17 14:33:15.263743 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.263680    2570 scope.go:117] "RemoveContainer" containerID="0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"
Sep 17 14:33:15.274228 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.274209    2570 scope.go:117] "RemoveContainer" containerID="0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"
Sep 17 14:33:15.274522 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:33:15.274495    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56\": container with ID starting with 0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56 not found: ID does not exist" containerID="0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"
Sep 17 14:33:15.274590 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.274535    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56"} err="failed to get container status \"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56\": rpc error: code = NotFound desc = could not find container \"0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56\": container with ID starting with 0096d87df65d2dc3845e4035d7b6735f426904810ba8d54dc365bb1b1020be56 not found: ID does not exist"
Sep 17 14:33:15.286741 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.286699    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:33:15.290868 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.290843    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-166f16-5777cbb8f8-42pvq"]
Sep 17 14:33:15.302113 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.302089    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:33:15.305716 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:15.305694    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-166f16-676b8dd896-d57wb"]
Sep 17 14:33:16.182139 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:16.182110    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9654db8-a35d-44fd-8da9-38728ebdc944" path="/var/lib/kubelet/pods/a9654db8-a35d-44fd-8da9-38728ebdc944/volumes"
Sep 17 14:33:16.182497 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:16.182484    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d68b7d34-2287-4ee9-9113-99527ae89b46" path="/var/lib/kubelet/pods/d68b7d34-2287-4ee9-9113-99527ae89b46/volumes"
Sep 17 14:33:17.707660 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:17.707619    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:33:32.546515 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.546480    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:33:32.546964 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.546768    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" podUID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" containerName="payload-processing" containerID="cri-o://82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c" gracePeriod=30
Sep 17 14:33:32.796605 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.796544    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:33:32.879400 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.879355    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpsn4\" (UniqueName: \"kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4\") pod \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") "
Sep 17 14:33:32.879586 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.879407    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume\") pod \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\" (UID: \"34bb8d85-0ece-43e4-8ac5-c08ef07a9170\") "
Sep 17 14:33:32.879818 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.879793    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34bb8d85-0ece-43e4-8ac5-c08ef07a9170" (UID: "34bb8d85-0ece-43e4-8ac5-c08ef07a9170"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:32.881631 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.881611    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4" (OuterVolumeSpecName: "kube-api-access-rpsn4") pod "34bb8d85-0ece-43e4-8ac5-c08ef07a9170" (UID: "34bb8d85-0ece-43e4-8ac5-c08ef07a9170"). InnerVolumeSpecName "kube-api-access-rpsn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:32.980240 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.980205    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:32.980240 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:32.980234    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpsn4\" (UniqueName: \"kubernetes.io/projected/34bb8d85-0ece-43e4-8ac5-c08ef07a9170-kube-api-access-rpsn4\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:33.358813 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.358774    2570 generic.go:356] "Generic (PLEG): container finished" podID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" containerID="82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c" exitCode=0
Sep 17 14:33:33.358979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.358846    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" event={"ID":"34bb8d85-0ece-43e4-8ac5-c08ef07a9170","Type":"ContainerDied","Data":"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"}
Sep 17 14:33:33.358979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.358871    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"
Sep 17 14:33:33.358979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.358899    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf" event={"ID":"34bb8d85-0ece-43e4-8ac5-c08ef07a9170","Type":"ContainerDied","Data":"d5bbbb1623c43c1bc7daeb3f0756e383c0336db537987ca6dc52bb64caa26026"}
Sep 17 14:33:33.358979 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.358928    2570 scope.go:117] "RemoveContainer" containerID="82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"
Sep 17 14:33:33.368415 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.368397    2570 scope.go:117] "RemoveContainer" containerID="82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"
Sep 17 14:33:33.368710 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:33:33.368689    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c\": container with ID starting with 82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c not found: ID does not exist" containerID="82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"
Sep 17 14:33:33.368841 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.368719    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c"} err="failed to get container status \"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c\": rpc error: code = NotFound desc = could not find container \"82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c\": container with ID starting with 82e017f9d7571e425f8ca0bd7f4c9ce84e8e97fdc131e6899e20a9610de25c6c not found: ID does not exist"
Sep 17 14:33:33.384726 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.384701    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:33:33.386930 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:33.386909    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-b88b90-79b4d8b6f-hvgxf"]
Sep 17 14:33:34.181158 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:34.181122    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" path="/var/lib/kubelet/pods/34bb8d85-0ece-43e4-8ac5-c08ef07a9170/volumes"
Sep 17 14:33:35.531216 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:35.531182    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:33:40.289359 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:40.289328    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:33:40.289851 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:40.289620    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy" containerID="cri-o://8a83b04362fcc105f9b8c60f2206cac75d83dce3356f874b2211332f311bf072" gracePeriod=30
Sep 17 14:33:45.425156 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.425124    2570 generic.go:356] "Generic (PLEG): container finished" podID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerID="8a83b04362fcc105f9b8c60f2206cac75d83dce3356f874b2211332f311bf072" exitCode=0
Sep 17 14:33:45.425609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.425174    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerDied","Data":"8a83b04362fcc105f9b8c60f2206cac75d83dce3356f874b2211332f311bf072"}
Sep 17 14:33:45.425609 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.425227    2570 scope.go:117] "RemoveContainer" containerID="0d5777be4a39449caba0a8a0e7b3626f04fa24ecfa620aab2cb7caa57521407d"
Sep 17 14:33:45.532302 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.532273    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:33:45.593785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593654    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.593785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593707    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.593785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593748    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.593785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593778    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593807    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqq6k\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593909    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593941    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593972    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.593993    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data" (OuterVolumeSpecName: "istio-data") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594007    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594042    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert\") pod \"37f20f97-1e63-4821-9d54-874aab9eeaec\" (UID: \"37f20f97-1e63-4821-9d54-874aab9eeaec\") "
Sep 17 14:33:45.594136 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594113    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594263    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594390    2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-data\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594410    2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-credential-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594424    2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594437    2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-workload-socket\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.594526 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.594473    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:45.596279 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.596253    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:33:45.596376 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.596273    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k" (OuterVolumeSpecName: "kube-api-access-hqq6k") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "kube-api-access-hqq6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:45.596767 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.596718    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token" (OuterVolumeSpecName: "istio-token") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:45.596767 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.596718    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37f20f97-1e63-4821-9d54-874aab9eeaec" (UID: "37f20f97-1e63-4821-9d54-874aab9eeaec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 17 14:33:45.695689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.695653    2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-podinfo\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.695689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.695685    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqq6k\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-kube-api-access-hqq6k\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.695689 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.695697    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.695946 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.695705    2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f20f97-1e63-4821-9d54-874aab9eeaec-istio-envoy\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:45.695946 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:45.695713    2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f20f97-1e63-4821-9d54-874aab9eeaec-istiod-ca-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:46.431333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:46.431299    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"
Sep 17 14:33:46.431333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:46.431311    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l" event={"ID":"37f20f97-1e63-4821-9d54-874aab9eeaec","Type":"ContainerDied","Data":"3d55ee0bba58f8da5abe5a154abf88454ebed8717931318732315c9aca31becc"}
Sep 17 14:33:46.431829 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:46.431374    2570 scope.go:117] "RemoveContainer" containerID="8a83b04362fcc105f9b8c60f2206cac75d83dce3356f874b2211332f311bf072"
Sep 17 14:33:46.449925 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:46.449901    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:33:46.456125 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:46.456100    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b88b90-openshift-default-7dc4ffc754-qh27l"]
Sep 17 14:33:48.181265 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:48.181230    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" path="/var/lib/kubelet/pods/37f20f97-1e63-4821-9d54-874aab9eeaec/volumes"
Sep 17 14:33:51.352932 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.352902    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:33:51.353410 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.353168    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" podUID="5d310bf0-958d-4797-93b1-434b7ff189a0" containerName="maas-api" containerID="cri-o://43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34" gracePeriod=30
Sep 17 14:33:51.371335 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.371303    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"]
Sep 17 14:33:51.384765 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.384709    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-b79988-29827590-448qv"]
Sep 17 14:33:51.418150 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.418115    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:33:51.418461 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.418409    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" podUID="531ead4a-772b-43f4-8642-26530715cddf" containerName="payload-pre-processing" containerID="cri-o://9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c" gracePeriod=30
Sep 17 14:33:51.692409 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.692381    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:33:51.696486 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.696464    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:33:51.757424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757381    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs\") pod \"5d310bf0-958d-4797-93b1-434b7ff189a0\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") "
Sep 17 14:33:51.757424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757419    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8p67\" (UniqueName: \"kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67\") pod \"531ead4a-772b-43f4-8642-26530715cddf\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") "
Sep 17 14:33:51.757677 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757456    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qx9n\" (UniqueName: \"kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n\") pod \"5d310bf0-958d-4797-93b1-434b7ff189a0\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") "
Sep 17 14:33:51.757677 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757474    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume\") pod \"531ead4a-772b-43f4-8642-26530715cddf\" (UID: \"531ead4a-772b-43f4-8642-26530715cddf\") "
Sep 17 14:33:51.757677 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757510    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca\") pod \"5d310bf0-958d-4797-93b1-434b7ff189a0\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") "
Sep 17 14:33:51.757677 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.757542    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") pod \"5d310bf0-958d-4797-93b1-434b7ff189a0\" (UID: \"5d310bf0-958d-4797-93b1-434b7ff189a0\") "
Sep 17 14:33:51.758149 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.758121    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5d310bf0-958d-4797-93b1-434b7ff189a0" (UID: "5d310bf0-958d-4797-93b1-434b7ff189a0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:51.758468 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.758436    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "531ead4a-772b-43f4-8642-26530715cddf" (UID: "531ead4a-772b-43f4-8642-26530715cddf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:33:51.760220 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.760197    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5d310bf0-958d-4797-93b1-434b7ff189a0" (UID: "5d310bf0-958d-4797-93b1-434b7ff189a0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:51.760220 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.760202    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67" (OuterVolumeSpecName: "kube-api-access-p8p67") pod "531ead4a-772b-43f4-8642-26530715cddf" (UID: "531ead4a-772b-43f4-8642-26530715cddf"). InnerVolumeSpecName "kube-api-access-p8p67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:51.760387 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.760370    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n" (OuterVolumeSpecName: "kube-api-access-7qx9n") pod "5d310bf0-958d-4797-93b1-434b7ff189a0" (UID: "5d310bf0-958d-4797-93b1-434b7ff189a0"). InnerVolumeSpecName "kube-api-access-7qx9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:33:51.760429 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.760387    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5d310bf0-958d-4797-93b1-434b7ff189a0" (UID: "5d310bf0-958d-4797-93b1-434b7ff189a0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:33:51.858671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858632    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:51.858671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858666    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8p67\" (UniqueName: \"kubernetes.io/projected/531ead4a-772b-43f4-8642-26530715cddf-kube-api-access-p8p67\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:51.858671 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858678    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qx9n\" (UniqueName: \"kubernetes.io/projected/5d310bf0-958d-4797-93b1-434b7ff189a0-kube-api-access-7qx9n\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:51.858954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858691    2570 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/531ead4a-772b-43f4-8642-26530715cddf-plugins-config-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:51.858954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858702    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d310bf0-958d-4797-93b1-434b7ff189a0-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:51.858954 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:51.858713    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d310bf0-958d-4797-93b1-434b7ff189a0-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:33:52.181368 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.181334    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7db7a284-05a0-48b3-8fef-6e5922ce10e5" path="/var/lib/kubelet/pods/7db7a284-05a0-48b3-8fef-6e5922ce10e5/volumes"
Sep 17 14:33:52.466485 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.466391    2570 generic.go:356] "Generic (PLEG): container finished" podID="5d310bf0-958d-4797-93b1-434b7ff189a0" containerID="43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34" exitCode=0
Sep 17 14:33:52.466485 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.466439    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"
Sep 17 14:33:52.467016 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.466486    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" event={"ID":"5d310bf0-958d-4797-93b1-434b7ff189a0","Type":"ContainerDied","Data":"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"}
Sep 17 14:33:52.467016 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.466543    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l" event={"ID":"5d310bf0-958d-4797-93b1-434b7ff189a0","Type":"ContainerDied","Data":"48f77743af2f23446f714f1f5c455d179eac40bc25319148b318fa216d35478f"}
Sep 17 14:33:52.467016 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.466574    2570 scope.go:117] "RemoveContainer" containerID="43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"
Sep 17 14:33:52.468186 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.468163    2570 generic.go:356] "Generic (PLEG): container finished" podID="531ead4a-772b-43f4-8642-26530715cddf" containerID="9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c" exitCode=0
Sep 17 14:33:52.468315 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.468193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" event={"ID":"531ead4a-772b-43f4-8642-26530715cddf","Type":"ContainerDied","Data":"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"}
Sep 17 14:33:52.468315 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.468220    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"
Sep 17 14:33:52.468315 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.468251    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm" event={"ID":"531ead4a-772b-43f4-8642-26530715cddf","Type":"ContainerDied","Data":"4495dd5e53cf2087db6742fafafb95ff8a7d515912d879bb75617efd670f8f54"}
Sep 17 14:33:52.475653 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.475635    2570 scope.go:117] "RemoveContainer" containerID="43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"
Sep 17 14:33:52.475976 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:33:52.475957    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34\": container with ID starting with 43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34 not found: ID does not exist" containerID="43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"
Sep 17 14:33:52.476027 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.475984    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34"} err="failed to get container status \"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34\": rpc error: code = NotFound desc = could not find container \"43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34\": container with ID starting with 43fcbf25a3d43ba41187af1508cc2d4b60dd7d603e99ad6d659909c711dace34 not found: ID does not exist"
Sep 17 14:33:52.476027 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.476001    2570 scope.go:117] "RemoveContainer" containerID="9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"
Sep 17 14:33:52.484024 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.484004    2570 scope.go:117] "RemoveContainer" containerID="9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"
Sep 17 14:33:52.484329 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:33:52.484309    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c\": container with ID starting with 9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c not found: ID does not exist" containerID="9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"
Sep 17 14:33:52.484378 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.484343    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c"} err="failed to get container status \"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c\": rpc error: code = NotFound desc = could not find container \"9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c\": container with ID starting with 9460688f74054989a8b8455691b2b550f4c84ac2ca358636b5e34d78a53d6f1c not found: ID does not exist"
Sep 17 14:33:52.489588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.489556    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:33:52.494690 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.494668    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b79988-5bc6689f64-4qz5l"]
Sep 17 14:33:52.507217 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.507195    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:33:52.510697 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:52.510677    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b79988-fff9dd8c4-5kgkm"]
Sep 17 14:33:54.119310 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:54.119276    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:33:54.182548 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:54.182510    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="531ead4a-772b-43f4-8642-26530715cddf" path="/var/lib/kubelet/pods/531ead4a-772b-43f4-8642-26530715cddf/volumes"
Sep 17 14:33:54.183166 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:33:54.183148    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d310bf0-958d-4797-93b1-434b7ff189a0" path="/var/lib/kubelet/pods/5d310bf0-958d-4797-93b1-434b7ff189a0/volumes"
Sep 17 14:34:04.742134 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:34:04.742092    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:35:25.621792 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:35:25.621755    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:35:50.620354 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:35:50.620315    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:35:59.214366 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:35:59.214329    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:36:08.230875 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:08.230838    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:36:15.918493 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:15.918454    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:36:29.227871 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:29.227832    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:36:35.840873 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:35.840837    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:36:39.091431 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.091393    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:36:39.091989 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.091661    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager" containerID="cri-o://86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5" gracePeriod=10
Sep 17 14:36:39.447383 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.447355    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:36:39.545135 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.545099    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume\") pod \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") "
Sep 17 14:36:39.545135 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.545132    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgd68\" (UniqueName: \"kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68\") pod \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\" (UID: \"ec3c4293-8e10-45da-92e3-bbeb21be55d5\") "
Sep 17 14:36:39.545506 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.545487    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ec3c4293-8e10-45da-92e3-bbeb21be55d5" (UID: "ec3c4293-8e10-45da-92e3-bbeb21be55d5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:36:39.547304 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.547282    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68" (OuterVolumeSpecName: "kube-api-access-sgd68") pod "ec3c4293-8e10-45da-92e3-bbeb21be55d5" (UID: "ec3c4293-8e10-45da-92e3-bbeb21be55d5"). InnerVolumeSpecName "kube-api-access-sgd68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:36:39.646041 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.645955    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ec3c4293-8e10-45da-92e3-bbeb21be55d5-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:36:39.646041 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:39.645986    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgd68\" (UniqueName: \"kubernetes.io/projected/ec3c4293-8e10-45da-92e3-bbeb21be55d5-kube-api-access-sgd68\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:36:40.280614 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.280576    2570 generic.go:356] "Generic (PLEG): container finished" podID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerID="86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5" exitCode=0
Sep 17 14:36:40.281044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.280639    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"
Sep 17 14:36:40.281044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.280671    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerDied","Data":"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"}
Sep 17 14:36:40.281044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.280750    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9" event={"ID":"ec3c4293-8e10-45da-92e3-bbeb21be55d5","Type":"ContainerDied","Data":"85a9bee9396f9bd430f4c32592677852bc202a25c3f9b1e24c6480eea8304100"}
Sep 17 14:36:40.281044 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.280757    2570 scope.go:117] "RemoveContainer" containerID="86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"
Sep 17 14:36:40.289647 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.289622    2570 scope.go:117] "RemoveContainer" containerID="403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"
Sep 17 14:36:40.297642 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.297624    2570 scope.go:117] "RemoveContainer" containerID="86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"
Sep 17 14:36:40.297976 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:36:40.297956    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5\": container with ID starting with 86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5 not found: ID does not exist" containerID="86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"
Sep 17 14:36:40.298035 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.297986    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5"} err="failed to get container status \"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5\": rpc error: code = NotFound desc = could not find container \"86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5\": container with ID starting with 86ec21b526ba619af6321cf13d3c4a1d0bc2d97c300a88354bc8446a1e65aeb5 not found: ID does not exist"
Sep 17 14:36:40.298035 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.298004    2570 scope.go:117] "RemoveContainer" containerID="403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"
Sep 17 14:36:40.298264 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:36:40.298249    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292\": container with ID starting with 403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292 not found: ID does not exist" containerID="403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"
Sep 17 14:36:40.298312 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.298267    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292"} err="failed to get container status \"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292\": rpc error: code = NotFound desc = could not find container \"403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292\": container with ID starting with 403da8370ecb14b2d6bdbe5ba709f186b6f5c5d65d707241193c58edfbc1b292 not found: ID does not exist"
Sep 17 14:36:40.302079 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.302059    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:36:40.308808 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:40.308787    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-2hxc9"]
Sep 17 14:36:42.180822 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:42.180790    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" path="/var/lib/kubelet/pods/ec3c4293-8e10-45da-92e3-bbeb21be55d5/volumes"
Sep 17 14:36:58.642464 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:58.642430    2570 scope.go:117] "RemoveContainer" containerID="adde01262fbb81eb4679e8f5ede7bc440057faa857a471300ffde48921337b03"
Sep 17 14:36:58.650743 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:36:58.650707    2570 scope.go:117] "RemoveContainer" containerID="61db8d19387c30d84d7f17a82e5a3f01d05e147d0e544e927d69caee37eff903"
Sep 17 14:37:39.786080 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786043    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"]
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786369    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786380    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786389    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9654db8-a35d-44fd-8da9-38728ebdc944" containerName="payload-pre-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786395    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9654db8-a35d-44fd-8da9-38728ebdc944" containerName="payload-pre-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786402    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63d6a177-3f10-469f-a763-ebef283ecd41" containerName="payload-pre-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786408    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="63d6a177-3f10-469f-a763-ebef283ecd41" containerName="payload-pre-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786417    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68b7d34-2287-4ee9-9113-99527ae89b46" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786422    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68b7d34-2287-4ee9-9113-99527ae89b46" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786429    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" containerName="payload-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786434    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" containerName="payload-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786444    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786450    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786460    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2de614d-444a-4728-821b-860e4dd755be" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786464    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2de614d-444a-4728-821b-860e4dd755be" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786473    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e160003-d84a-4f71-8648-6880443969e8" containerName="payload-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786478    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e160003-d84a-4f71-8648-6880443969e8" containerName="payload-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786484    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d310bf0-958d-4797-93b1-434b7ff189a0" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786489    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d310bf0-958d-4797-93b1-434b7ff189a0" containerName="maas-api"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786496    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786502    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786509    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="531ead4a-772b-43f4-8642-26530715cddf" containerName="payload-pre-processing"
Sep 17 14:37:39.786537 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786514    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="531ead4a-772b-43f4-8642-26530715cddf" containerName="payload-pre-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786562    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786570    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d310bf0-958d-4797-93b1-434b7ff189a0" containerName="maas-api"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786576    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68b7d34-2287-4ee9-9113-99527ae89b46" containerName="maas-api"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786581    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="63d6a177-3f10-469f-a763-ebef283ecd41" containerName="payload-pre-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786587    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9654db8-a35d-44fd-8da9-38728ebdc944" containerName="payload-pre-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786592    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="34bb8d85-0ece-43e4-8ac5-c08ef07a9170" containerName="payload-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786599    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786603    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="531ead4a-772b-43f4-8642-26530715cddf" containerName="payload-pre-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786608    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786614    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e160003-d84a-4f71-8648-6880443969e8" containerName="payload-processing"
Sep 17 14:37:39.787239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.786620    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2de614d-444a-4728-821b-860e4dd755be" containerName="maas-api"
Sep 17 14:37:39.789459 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.789441    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.792309 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.792289    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xbqnv\""
Sep 17 14:37:39.802724 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.802696    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"]
Sep 17 14:37:39.879458 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.879419    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.879652 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.879548    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdrf\" (UniqueName: \"kubernetes.io/projected/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-kube-api-access-zcdrf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.980297 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.980252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdrf\" (UniqueName: \"kubernetes.io/projected/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-kube-api-access-zcdrf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.980512 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.980326    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.980724 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.980704    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:39.991398 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:39.991371    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdrf\" (UniqueName: \"kubernetes.io/projected/86efb2e0-a8f8-4eca-9a24-9ea74f9e207c-kube-api-access-zcdrf\") pod \"kuadrant-operator-controller-manager-5769f99d9f-vn8fz\" (UID: \"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:40.099052 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.098946    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:40.387032 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.387012    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"]
Sep 17 14:37:40.389358 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:37:40.389326    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86efb2e0_a8f8_4eca_9a24_9ea74f9e207c.slice/crio-a9bc9701b7abccf9ed80935de0207770fd3e141b1a1728cf1855de7c23b78fe0 WatchSource:0}: Error finding container a9bc9701b7abccf9ed80935de0207770fd3e141b1a1728cf1855de7c23b78fe0: Status 404 returned error can't find the container with id a9bc9701b7abccf9ed80935de0207770fd3e141b1a1728cf1855de7c23b78fe0
Sep 17 14:37:40.564208 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.564173    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz" event={"ID":"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c","Type":"ContainerStarted","Data":"25044fd9806d924ebe7a3c4199a687493c324351a4ea883d6312741e18072539"}
Sep 17 14:37:40.564386 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.564233    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz" event={"ID":"86efb2e0-a8f8-4eca-9a24-9ea74f9e207c","Type":"ContainerStarted","Data":"a9bc9701b7abccf9ed80935de0207770fd3e141b1a1728cf1855de7c23b78fe0"}
Sep 17 14:37:40.564386 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.564358    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:37:40.592307 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:40.592254    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz" podStartSLOduration=1.592236202 podStartE2EDuration="1.592236202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:37:40.58994751 +0000 UTC m=+2502.826824989" watchObservedRunningTime="2026-09-17 14:37:40.592236202 +0000 UTC m=+2502.829113678"
Sep 17 14:37:51.570877 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:37:51.570793    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-vn8fz"
Sep 17 14:38:05.317844 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:05.317793    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:13.933850 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:13.933814    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:22.318497 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.318461    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:38:22.318955 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.318822    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:38:22.318955 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.318835    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec3c4293-8e10-45da-92e3-bbeb21be55d5" containerName="manager"
Sep 17 14:38:22.318955 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.318891    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f20f97-1e63-4821-9d54-874aab9eeaec" containerName="istio-proxy"
Sep 17 14:38:22.322231 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.322211    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.330686 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.330667    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d3c310b5-kserve-self-signed-certs\""
Sep 17 14:38:22.457986 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.457951    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:38:22.462655 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462633    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j455f\" (UniqueName: \"kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.462783 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.462783 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.462783 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462777    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.462916 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462812    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.462916 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.462865    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563573 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j455f\" (UniqueName: \"kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563584    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563602    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563642    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.563785 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.563717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.564062 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.564039    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.564114 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.564092    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.564156 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.564110    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.566091 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.566069    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.566358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.566340    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.586327 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.586270    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j455f\" (UniqueName: \"kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f\") pod \"e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") " pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.631331 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.631298    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:22.907248 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:22.907224    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:38:22.909463 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:38:22.909440    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30ee70b4_ea76_458d_ab2c_3016d4fc2f17.slice/crio-cf78f02ee6e6fa415149f4839d2c0a91bb6bf0c3d02bfe588f00547dee9944ba WatchSource:0}: Error finding container cf78f02ee6e6fa415149f4839d2c0a91bb6bf0c3d02bfe588f00547dee9944ba: Status 404 returned error can't find the container with id cf78f02ee6e6fa415149f4839d2c0a91bb6bf0c3d02bfe588f00547dee9944ba
Sep 17 14:38:23.770172 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:23.770137    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" event={"ID":"30ee70b4-ea76-458d-ab2c-3016d4fc2f17","Type":"ContainerStarted","Data":"8f6af30e61815f5aa62da8252e79f53cee1c3d208c818e32a74bcc40276cde30"}
Sep 17 14:38:23.770172 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:23.770178    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" event={"ID":"30ee70b4-ea76-458d-ab2c-3016d4fc2f17","Type":"ContainerStarted","Data":"cf78f02ee6e6fa415149f4839d2c0a91bb6bf0c3d02bfe588f00547dee9944ba"}
Sep 17 14:38:23.792725 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:23.792677    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" podStartSLOduration=1.792662406 podStartE2EDuration="1.792662406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:38:23.791063448 +0000 UTC m=+2546.027940926" watchObservedRunningTime="2026-09-17 14:38:23.792662406 +0000 UTC m=+2546.029539904"
Sep 17 14:38:25.421864 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:25.421831    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:32.632097 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:32.632050    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:32.632507 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:32.632114    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:32.644699 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:32.644671    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:32.828685 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:32.828650    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:38:36.514627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:36.514594    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:47.789279 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:47.789239    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:38:50.018604 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:50.018566    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:58.339333 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.339297    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:38:58.342685 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.342667    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.346840 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.346811    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 17 14:38:58.347071 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.347053    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-j4j8k\""
Sep 17 14:38:58.347178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.347077    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 17 14:38:58.347178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.347079    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 17 14:38:58.347178 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.347160    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 17 14:38:58.355150 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.355123    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:38:58.386107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.386071    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.386285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.386139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.386285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.386180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.386285 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.386202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jvzr\" (UniqueName: \"kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.487580 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.487527    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.487580 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.487578    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jvzr\" (UniqueName: \"kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.487885 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.487620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.487885 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.487665    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.488242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.488212    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.490239 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.490217    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.490329 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.490306    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.497457 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.497429    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jvzr\" (UniqueName: \"kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr\") pod \"maas-api-5f544fdfdc-zzf7p\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") " pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.653822 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.653787    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:58.935491 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.935463    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:38:58.938087 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:38:58.938047    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4a102d_217b_4c62_9a75_843754f174d9.slice/crio-6ad96cc18441c1fc9d618db6312992230e250e45b54a30f16b22fcbe61d970b5 WatchSource:0}: Error finding container 6ad96cc18441c1fc9d618db6312992230e250e45b54a30f16b22fcbe61d970b5: Status 404 returned error can't find the container with id 6ad96cc18441c1fc9d618db6312992230e250e45b54a30f16b22fcbe61d970b5
Sep 17 14:38:58.939530 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.939511    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 14:38:58.945661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:58.945640    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" event={"ID":"1a4a102d-217b-4c62-9a75-843754f174d9","Type":"ContainerStarted","Data":"6ad96cc18441c1fc9d618db6312992230e250e45b54a30f16b22fcbe61d970b5"}
Sep 17 14:38:59.499070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:59.499039    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:38:59.953561 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:59.953525    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" event={"ID":"1a4a102d-217b-4c62-9a75-843754f174d9","Type":"ContainerStarted","Data":"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"}
Sep 17 14:38:59.953800 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:59.953627    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:38:59.973358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:38:59.973183    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" podStartSLOduration=1.532103662 podStartE2EDuration="1.97316595s" totalImagesPullingTime="441.062288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:38:59.972258435 +0000 UTC m=+2582.209135912" watchObservedRunningTime="2026-09-17 14:38:59.97316595 +0000 UTC m=+2582.210043427"
Sep 17 14:39:02.809174 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:02.809126    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerName="main" containerID="cri-o://8f6af30e61815f5aa62da8252e79f53cee1c3d208c818e32a74bcc40276cde30" gracePeriod=45
Sep 17 14:39:02.817948 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:02.817914    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerName="main" probeResult="failure" output="Get \"https://10.134.0.87:8000/ready\": dial tcp 10.134.0.87:8000: connect: connection refused"
Sep 17 14:39:02.980863 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:02.980832    2570 generic.go:356] "Generic (PLEG): container finished" podID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerID="8f6af30e61815f5aa62da8252e79f53cee1c3d208c818e32a74bcc40276cde30" exitCode=0
Sep 17 14:39:02.981018 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:02.980914    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" event={"ID":"30ee70b4-ea76-458d-ab2c-3016d4fc2f17","Type":"ContainerDied","Data":"8f6af30e61815f5aa62da8252e79f53cee1c3d208c818e32a74bcc40276cde30"}
Sep 17 14:39:03.061499 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.061428    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:39:03.127475 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127443    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127486    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127527    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127560    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127590    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127661 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127607    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j455f\" (UniqueName: \"kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f\") pod \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\" (UID: \"30ee70b4-ea76-458d-ab2c-3016d4fc2f17\") "
Sep 17 14:39:03.127961 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127762    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home" (OuterVolumeSpecName: "home") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:39:03.127961 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127857    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache" (OuterVolumeSpecName: "model-cache") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:39:03.127961 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.127946    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.128079 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.128036    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:39:03.130068 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.130044    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm" (OuterVolumeSpecName: "dshm") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 14:39:03.130152 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.130099    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f" (OuterVolumeSpecName: "kube-api-access-j455f") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "kube-api-access-j455f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:39:03.130152 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.130136    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30ee70b4-ea76-458d-ab2c-3016d4fc2f17" (UID: "30ee70b4-ea76-458d-ab2c-3016d4fc2f17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:39:03.228466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.228432    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.228466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.228465    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.228466 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.228475    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.228723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.228482    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.228723 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.228493    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j455f\" (UniqueName: \"kubernetes.io/projected/30ee70b4-ea76-458d-ab2c-3016d4fc2f17-kube-api-access-j455f\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:03.987058 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.987021    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"
Sep 17 14:39:03.987482 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.987073    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv" event={"ID":"30ee70b4-ea76-458d-ab2c-3016d4fc2f17","Type":"ContainerDied","Data":"cf78f02ee6e6fa415149f4839d2c0a91bb6bf0c3d02bfe588f00547dee9944ba"}
Sep 17 14:39:03.987482 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:03.987123    2570 scope.go:117] "RemoveContainer" containerID="8f6af30e61815f5aa62da8252e79f53cee1c3d208c818e32a74bcc40276cde30"
Sep 17 14:39:04.010810 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:04.010781    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:39:04.017518 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:04.017492    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d3c310b5-kserve-6cb8458f87-hfmjv"]
Sep 17 14:39:04.182410 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:04.182373    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" path="/var/lib/kubelet/pods/30ee70b4-ea76-458d-ab2c-3016d4fc2f17/volumes"
Sep 17 14:39:05.587266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.587231    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:39:05.587712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.587433    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" podUID="1a4a102d-217b-4c62-9a75-843754f174d9" containerName="maas-api" containerID="cri-o://8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3" gracePeriod=30
Sep 17 14:39:05.592867 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.592839    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:39:05.624752 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.624697    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74ddfb899-hws95"]
Sep 17 14:39:05.625107 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.625094    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerName="main"
Sep 17 14:39:05.625163 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.625110    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerName="main"
Sep 17 14:39:05.625199 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.625180    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ee70b4-ea76-458d-ab2c-3016d4fc2f17" containerName="main"
Sep 17 14:39:05.628063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.628044    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.645203 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.645173    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74ddfb899-hws95"]
Sep 17 14:39:05.749599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.749558    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-maas-api-tls\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.749808 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.749644    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5pjp\" (UniqueName: \"kubernetes.io/projected/9277ab21-ae8a-46c5-b704-330ad23bd773-kube-api-access-q5pjp\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.749808 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.749716    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-metrics-certs\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.749940 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.749861    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9277ab21-ae8a-46c5-b704-330ad23bd773-openshift-service-ca\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.843564 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.843496    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:39:05.850771 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.850725    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9277ab21-ae8a-46c5-b704-330ad23bd773-openshift-service-ca\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.850915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.850813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-maas-api-tls\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.850915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.850862    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5pjp\" (UniqueName: \"kubernetes.io/projected/9277ab21-ae8a-46c5-b704-330ad23bd773-kube-api-access-q5pjp\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.850915 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.850901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-metrics-certs\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.851510 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.851447    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9277ab21-ae8a-46c5-b704-330ad23bd773-openshift-service-ca\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.853443 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.853423    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-maas-api-tls\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.853554 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.853444    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9277ab21-ae8a-46c5-b704-330ad23bd773-metrics-certs\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.860464 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.860428    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5pjp\" (UniqueName: \"kubernetes.io/projected/9277ab21-ae8a-46c5-b704-330ad23bd773-kube-api-access-q5pjp\") pod \"maas-api-74ddfb899-hws95\" (UID: \"9277ab21-ae8a-46c5-b704-330ad23bd773\") " pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.952105 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.952068    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca\") pod \"1a4a102d-217b-4c62-9a75-843754f174d9\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") "
Sep 17 14:39:05.952305 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.952114    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs\") pod \"1a4a102d-217b-4c62-9a75-843754f174d9\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") "
Sep 17 14:39:05.952305 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.952160    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jvzr\" (UniqueName: \"kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr\") pod \"1a4a102d-217b-4c62-9a75-843754f174d9\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") "
Sep 17 14:39:05.952305 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.952247    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls\") pod \"1a4a102d-217b-4c62-9a75-843754f174d9\" (UID: \"1a4a102d-217b-4c62-9a75-843754f174d9\") "
Sep 17 14:39:05.952543 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.952513    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1a4a102d-217b-4c62-9a75-843754f174d9" (UID: "1a4a102d-217b-4c62-9a75-843754f174d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 14:39:05.954588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.954556    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1a4a102d-217b-4c62-9a75-843754f174d9" (UID: "1a4a102d-217b-4c62-9a75-843754f174d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:39:05.954588 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.954581    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1a4a102d-217b-4c62-9a75-843754f174d9" (UID: "1a4a102d-217b-4c62-9a75-843754f174d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 14:39:05.954758 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.954580    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr" (OuterVolumeSpecName: "kube-api-access-7jvzr") pod "1a4a102d-217b-4c62-9a75-843754f174d9" (UID: "1a4a102d-217b-4c62-9a75-843754f174d9"). InnerVolumeSpecName "kube-api-access-7jvzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 14:39:05.965863 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.965825    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:05.998477 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.998445    2570 generic.go:356] "Generic (PLEG): container finished" podID="1a4a102d-217b-4c62-9a75-843754f174d9" containerID="8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3" exitCode=0
Sep 17 14:39:05.998620 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.998495    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"
Sep 17 14:39:05.998620 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.998528    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" event={"ID":"1a4a102d-217b-4c62-9a75-843754f174d9","Type":"ContainerDied","Data":"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"}
Sep 17 14:39:05.998620 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.998587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p" event={"ID":"1a4a102d-217b-4c62-9a75-843754f174d9","Type":"ContainerDied","Data":"6ad96cc18441c1fc9d618db6312992230e250e45b54a30f16b22fcbe61d970b5"}
Sep 17 14:39:05.998620 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:05.998617    2570 scope.go:117] "RemoveContainer" containerID="8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"
Sep 17 14:39:06.012830 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.012793    2570 scope.go:117] "RemoveContainer" containerID="8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"
Sep 17 14:39:06.013289 ip-10-0-130-163 kubenswrapper[2570]: E0917 14:39:06.013264    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3\": container with ID starting with 8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3 not found: ID does not exist" containerID="8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"
Sep 17 14:39:06.013424 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.013299    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3"} err="failed to get container status \"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3\": rpc error: code = NotFound desc = could not find container \"8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3\": container with ID starting with 8eed8bcff22d5c7cb7005bdf2c6d27d3c83a0fe3035710e66f51b9ba3c3f53d3 not found: ID does not exist"
Sep 17 14:39:06.024144 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.024111    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:39:06.030063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.030036    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5f544fdfdc-zzf7p"]
Sep 17 14:39:06.053376 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.053344    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:06.053376 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.053377    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a4a102d-217b-4c62-9a75-843754f174d9-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:06.053599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.053388    2570 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a4a102d-217b-4c62-9a75-843754f174d9-metrics-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:06.053599 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.053398    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jvzr\" (UniqueName: \"kubernetes.io/projected/1a4a102d-217b-4c62-9a75-843754f174d9-kube-api-access-7jvzr\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Sep 17 14:39:06.181917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.181882    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4a102d-217b-4c62-9a75-843754f174d9" path="/var/lib/kubelet/pods/1a4a102d-217b-4c62-9a75-843754f174d9/volumes"
Sep 17 14:39:06.256767 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:06.256722    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74ddfb899-hws95"]
Sep 17 14:39:06.260003 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:39:06.259973    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9277ab21_ae8a_46c5_b704_330ad23bd773.slice/crio-dbf5b9a2c1656274683e1b8a27be4dc237292ee23a58b716ded06228d4e9e040 WatchSource:0}: Error finding container dbf5b9a2c1656274683e1b8a27be4dc237292ee23a58b716ded06228d4e9e040: Status 404 returned error can't find the container with id dbf5b9a2c1656274683e1b8a27be4dc237292ee23a58b716ded06228d4e9e040
Sep 17 14:39:07.006511 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:07.006470    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95" event={"ID":"9277ab21-ae8a-46c5-b704-330ad23bd773","Type":"ContainerStarted","Data":"c9588b34f29de02bea2d547a5673ee63b1acb123a9abfee08b84ddf7e143822f"}
Sep 17 14:39:07.006957 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:07.006524    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95" event={"ID":"9277ab21-ae8a-46c5-b704-330ad23bd773","Type":"ContainerStarted","Data":"dbf5b9a2c1656274683e1b8a27be4dc237292ee23a58b716ded06228d4e9e040"}
Sep 17 14:39:07.006957 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:07.006617    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:07.027011 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:07.026978    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:39:07.030195 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:07.030153    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95" podStartSLOduration=1.609358867 podStartE2EDuration="2.030139091s" totalImagesPullingTime="420.780224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 14:39:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:39:07.029551831 +0000 UTC m=+2589.266429309" watchObservedRunningTime="2026-09-17 14:39:07.030139091 +0000 UTC m=+2589.267016576"
Sep 17 14:39:13.018906 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:13.018872    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74ddfb899-hws95"
Sep 17 14:39:14.916129 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:14.916085    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:39:20.930300 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:20.930261    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:39:28.268912 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:28.268874    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-lkk9s"]
Sep 17 14:39:30.530394 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:30.530362    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cf6768ff4-fj9t6_889e95ac-aaee-4a69-af31-faae509e2548/authorino/0.log"
Sep 17 14:39:36.268541 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:36.268508    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wb2tb_95743573-6428-45b9-b3e7-54d7c3ff7382/manager/0.log"
Sep 17 14:39:36.420271 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:36.420242    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xzhzb_a9211d7b-c78a-4266-b2b3-7e19b49cf686/manager/0.log"
Sep 17 14:39:37.091573 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:37.091540    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcc687555-fwkzn_8e45c1c6-390e-4a0a-9132-a9c0162b868a/copy-manifests/0.log"
Sep 17 14:39:37.114145 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:37.114112    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcc687555-fwkzn_8e45c1c6-390e-4a0a-9132-a9c0162b868a/manager/0.log"
Sep 17 14:39:37.714827 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:37.714795    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74ddfb899-hws95_9277ab21-ae8a-46c5-b704-330ad23bd773/maas-api/0.log"
Sep 17 14:39:38.832980 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:38.832929    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cf6768ff4-fj9t6_889e95ac-aaee-4a69-af31-faae509e2548/authorino/0.log"
Sep 17 14:39:38.957121 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:38.957090    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tfdzb_b1cc28a7-1085-4d55-abbd-18bcdd569f9c/manager/0.log"
Sep 17 14:39:39.078266 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:39.078235    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hdgbm_067db9d3-71d2-4051-ba7f-70a5d6bc96b3/manager/0.log"
Sep 17 14:39:39.209559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:39.209533    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-pqrlh_22b2221d-1485-40f0-bb8f-3ca3f1dd0dae/kuadrant-console-plugin/0.log"
Sep 17 14:39:39.350327 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:39.350300    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-vn8fz_86efb2e0-a8f8-4eca-9a24-9ea74f9e207c/manager/0.log"
Sep 17 14:39:39.484193 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:39.484113    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lkk9s_14547245-c2cf-403b-9e62-11154b55a41f/limitador/0.log"
Sep 17 14:39:39.605889 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:39.605847    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-wnzng_97ed6e53-800e-448a-87bd-d0a65f4d075b/manager/0.log"
Sep 17 14:39:40.278036 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:40.278006    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d75c69df-zhqld_c0ad5755-2296-4448-a37c-821c89dc7adc/kube-auth-proxy/0.log"
Sep 17 14:39:40.401451 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:40.401420    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/1.log"
Sep 17 14:39:41.353773 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:41.353721    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-m8qzz_830200e7-15e8-4dba-99be-0dd1cacf8f29/main/0.log"
Sep 17 14:39:41.856500 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:41.856469    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-btx67_e634a448-c17e-401f-bbc2-662720ec54da/main/0.log"
Sep 17 14:39:50.509823 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:50.509794    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vpfvq_46b9c53f-aba0-48fc-b364-d9c3a167f498/global-pull-secret-syncer/0.log"
Sep 17 14:39:50.675593 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:50.675559    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zjt8f_bdb2c331-e40e-42dd-963c-5b2230d90e5a/konnectivity-agent/0.log"
Sep 17 14:39:50.731800 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:50.731763    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-163.ec2.internal_c4f272aba07b562b95d8ca5dc3318b8a/haproxy/0.log"
Sep 17 14:39:55.807683 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:55.807628    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cf6768ff4-fj9t6_889e95ac-aaee-4a69-af31-faae509e2548/authorino/0.log"
Sep 17 14:39:55.859748 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:55.859697    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tfdzb_b1cc28a7-1085-4d55-abbd-18bcdd569f9c/manager/0.log"
Sep 17 14:39:55.898939 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:55.898892    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hdgbm_067db9d3-71d2-4051-ba7f-70a5d6bc96b3/manager/0.log"
Sep 17 14:39:55.930462 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:55.930436    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-pqrlh_22b2221d-1485-40f0-bb8f-3ca3f1dd0dae/kuadrant-console-plugin/0.log"
Sep 17 14:39:55.991584 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:55.991543    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-vn8fz_86efb2e0-a8f8-4eca-9a24-9ea74f9e207c/manager/0.log"
Sep 17 14:39:56.033246 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:56.033213    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-lkk9s_14547245-c2cf-403b-9e62-11154b55a41f/limitador/0.log"
Sep 17 14:39:56.130820 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:56.130701    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-wnzng_97ed6e53-800e-448a-87bd-d0a65f4d075b/manager/0.log"
Sep 17 14:39:58.398947 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:58.398915    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnc84_a13ff726-d8ff-495c-887f-8d1a90599cde/node-exporter/0.log"
Sep 17 14:39:58.423381 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:58.423353    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnc84_a13ff726-d8ff-495c-887f-8d1a90599cde/kube-rbac-proxy/0.log"
Sep 17 14:39:58.450252 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:58.450223    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qnc84_a13ff726-d8ff-495c-887f-8d1a90599cde/init-textfile/0.log"
Sep 17 14:39:59.590023 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.589989    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"]
Sep 17 14:39:59.590406 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.590337    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4a102d-217b-4c62-9a75-843754f174d9" containerName="maas-api"
Sep 17 14:39:59.590406 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.590348    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4a102d-217b-4c62-9a75-843754f174d9" containerName="maas-api"
Sep 17 14:39:59.590488 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.590445    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4a102d-217b-4c62-9a75-843754f174d9" containerName="maas-api"
Sep 17 14:39:59.594819 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.594800    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.600305 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.600282    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g9q75\"/\"kube-root-ca.crt\""
Sep 17 14:39:59.600423 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.600289    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g9q75\"/\"openshift-service-ca.crt\""
Sep 17 14:39:59.601512 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.601493    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g9q75\"/\"default-dockercfg-9jrgw\""
Sep 17 14:39:59.612627 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.612596    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"]
Sep 17 14:39:59.721521 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.721485    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-proc\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.721712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.721550    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-podres\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.721712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.721612    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-sys\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.721712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.721657    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-lib-modules\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.721712 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.721690    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98t5p\" (UniqueName: \"kubernetes.io/projected/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-kube-api-access-98t5p\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.822855 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822821    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-podres\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.822855 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822859    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-sys\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822884    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-lib-modules\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822915    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98t5p\" (UniqueName: \"kubernetes.io/projected/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-kube-api-access-98t5p\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822952    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-proc\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.822978    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-sys\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.823026    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-podres\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.823043    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-proc\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.823063 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.823043    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-lib-modules\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.835259 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.835235    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98t5p\" (UniqueName: \"kubernetes.io/projected/9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7-kube-api-access-98t5p\") pod \"perf-node-gather-daemonset-flkc4\" (UID: \"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7\") " pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:39:59.904401 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:39:59.904373    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:40:00.192559 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.192535    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"]
Sep 17 14:40:00.195327 ip-10-0-130-163 kubenswrapper[2570]: W0917 14:40:00.195261    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9636c4cf_f72a_4cdc_b1fd_9cc24c47c6d7.slice/crio-d69b11ec89a75fe41fce1da89145c8b921e885e6a4941e3f3a52cf57211a12d4 WatchSource:0}: Error finding container d69b11ec89a75fe41fce1da89145c8b921e885e6a4941e3f3a52cf57211a12d4: Status 404 returned error can't find the container with id d69b11ec89a75fe41fce1da89145c8b921e885e6a4941e3f3a52cf57211a12d4
Sep 17 14:40:00.283483 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.283446    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4" event={"ID":"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7","Type":"ContainerStarted","Data":"76e9565d9fb3ccb24a734786e37119c3442f4dff80901f50d5cdaf8485c62dd8"}
Sep 17 14:40:00.283665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.283508    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:40:00.283665 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.283526    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4" event={"ID":"9636c4cf-f72a-4cdc-b1fd-9cc24c47c6d7","Type":"ContainerStarted","Data":"d69b11ec89a75fe41fce1da89145c8b921e885e6a4941e3f3a52cf57211a12d4"}
Sep 17 14:40:00.302070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.302022    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4" podStartSLOduration=1.302006344 podStartE2EDuration="1.302006344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 14:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 14:40:00.300949671 +0000 UTC m=+2642.537827148" watchObservedRunningTime="2026-09-17 14:40:00.302006344 +0000 UTC m=+2642.538883820"
Sep 17 14:40:00.348417 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:00.348391    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-lwfc4_bcf448c1-5c33-4e1b-9f66-45ce62ac1e1b/networking-console-plugin/0.log"
Sep 17 14:40:02.426146 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:02.426109    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-m82gp_fe2e7469-1275-48b1-9b3a-84e1455e35f9/volume-data-source-validator/0.log"
Sep 17 14:40:03.670410 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:03.670379    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2j5sd_9c05b88d-95b9-4735-bfd6-f896023da4be/dns/0.log"
Sep 17 14:40:03.698202 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:03.698177    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2j5sd_9c05b88d-95b9-4735-bfd6-f896023da4be/kube-rbac-proxy/0.log"
Sep 17 14:40:03.919283 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:03.919256    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sw8gq_10ac6fa6-352e-4052-8d50-dfc868b889f0/dns-node-resolver/0.log"
Sep 17 14:40:04.571278 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:04.571245    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-844f746c6f-br9bn_8a31d439-e8b3-44e8-85e3-de77cf76ec55/registry/0.log"
Sep 17 14:40:04.650640 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:04.650603    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dbsps_ddc3a6a1-6e86-47ca-811a-4334fbcffbbb/node-ca/0.log"
Sep 17 14:40:06.299195 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:06.299170    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g9q75/perf-node-gather-daemonset-flkc4"
Sep 17 14:40:06.686582 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:06.686551    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d75c69df-zhqld_c0ad5755-2296-4448-a37c-821c89dc7adc/kube-auth-proxy/0.log"
Sep 17 14:40:06.717755 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:06.717708    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/0.log"
Sep 17 14:40:06.722299 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:06.722274    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/1.log"
Sep 17 14:40:07.805070 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:07.805044    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrswq_9e4abac5-542b-4c97-9875-df2a128c7d61/serve-healthcheck-canary/0.log"
Sep 17 14:40:08.408433 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:08.408402    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qssf9_5f60e758-62a8-4f6b-bc63-7366f201fa4c/insights-operator/0.log"
Sep 17 14:40:08.410917 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:08.410894    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-qssf9_5f60e758-62a8-4f6b-bc63-7366f201fa4c/insights-operator/1.log"
Sep 17 14:40:08.470222 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:08.470139    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22gkb_ec40f70f-5ed2-4631-b3dc-bf047ad23703/kube-rbac-proxy/0.log"
Sep 17 14:40:08.510024 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:08.509994    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22gkb_ec40f70f-5ed2-4631-b3dc-bf047ad23703/exporter/0.log"
Sep 17 14:40:08.568242 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:08.568208    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22gkb_ec40f70f-5ed2-4631-b3dc-bf047ad23703/extractor/0.log"
Sep 17 14:40:11.385931 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:11.385900    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d75c69df-zhqld_c0ad5755-2296-4448-a37c-821c89dc7adc/kube-auth-proxy/0.log"
Sep 17 14:40:11.411071 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:11.411038    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/0.log"
Sep 17 14:40:11.415394 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:11.415372    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/1.log"
Sep 17 14:40:11.559638 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:11.559607    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wb2tb_95743573-6428-45b9-b3e7-54d7c3ff7382/manager/0.log"
Sep 17 14:40:12.017901 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:12.017822    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xzhzb_a9211d7b-c78a-4266-b2b3-7e19b49cf686/manager/0.log"
Sep 17 14:40:12.491224 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:12.491186    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcc687555-fwkzn_8e45c1c6-390e-4a0a-9132-a9c0162b868a/manager/0.log"
Sep 17 14:40:12.649549 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:12.649519    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fcc687555-fwkzn_8e45c1c6-390e-4a0a-9132-a9c0162b868a/copy-manifests/0.log"
Sep 17 14:40:13.817358 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:13.817325    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-66d75c69df-zhqld_c0ad5755-2296-4448-a37c-821c89dc7adc/kube-auth-proxy/0.log"
Sep 17 14:40:13.873919 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:13.873884    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/0.log"
Sep 17 14:40:13.878528 ip-10-0-130-163 kubenswrapper[2570]: I0917 14:40:13.878493    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-lk9v4_957003c4-9cea-44ba-a8db-af7c2caf48b6/istio-proxy/1.log"