Sep 18 17:58:19.378242 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 17:58:19.378251 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 17:58:19.378259 ip-10-0-129-128 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 17:58:19.378410 ip-10-0-129-128 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 17:58:29.627497 ip-10-0-129-128 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 17:58:29.627517 ip-10-0-129-128 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 62d758dbb1f348f58353751e25faa5e0 --
Sep 18 18:00:26.644303 ip-10-0-129-128 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 18:00:30.484877 ip-10-0-129-128 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:30.484877 ip-10-0-129-128 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 18:00:30.484877 ip-10-0-129-128 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:30.484877 ip-10-0-129-128 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 18:00:30.484877 ip-10-0-129-128 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 18:00:30.485938 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.485822    2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 18:00:30.488955 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488939    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.488955 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488955    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488959    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488963    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488967    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488973    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488978    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488981    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488985    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488988    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488991    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488994    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488996    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.488999    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489003    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489006    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489025    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489030    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489034    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489038    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.489034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489043    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489047    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489057    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489061    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489067    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489070    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489073    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489076    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489079    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489082    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489085    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489088    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489091    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489093    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489096    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489099    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489101    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489105    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489107    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.489537 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489110    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489113    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489116    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489119    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489121    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489124    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489127    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489130    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489133    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489136    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489139    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489142    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489145    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489147    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489151    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489154    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489156    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489159    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489162    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489165    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.490085 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489168    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489170    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489173    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489176    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489179    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489182    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489185    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489188    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489191    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489194    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489197    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489200    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489202    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489205    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489208    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489211    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489213    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489216    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489220    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489223    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489226    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.490617 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489229    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489232    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489234    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489237    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489240    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489244    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489246    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489249    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489252    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489255    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489258    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489260    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489263    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489266    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489269    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489272    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489275    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489278    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489280    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.491238 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.489996    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490002    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490005    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490008    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490026    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490030    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490033    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490036    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490039    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490042    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490045    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490048    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490052    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490055    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490057    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490060    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490063    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490066    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490069    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490072    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.491739 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490075    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490078    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490081    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490084    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490087    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490090    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490094    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490097    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490099    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490102    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490105    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490108    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490111    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490113    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490116    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490119    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490121    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490124    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490127    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490130    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.492323 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490135    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490141    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490144    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490157    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490161    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490163    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490167    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490171    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490175    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490178    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490180    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490183    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490186    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490189    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490192    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490195    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490198    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490201    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490203    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.492896 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490207    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490209    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490212    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490215    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490218    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490220    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490223    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490226    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490228    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490231    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490234    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490237    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490242    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490244    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490248    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490250    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490253    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490256    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490259    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490263    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.493425 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490266    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490269    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490271    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490274    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490278    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490283    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490286    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490289    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490292    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490295    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490298    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490301    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490304    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490307    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490310    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490314    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490317    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490320    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490322    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.493953 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490326    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490714    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490719    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490722    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490726    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490729    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490732    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490736    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490739    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490741    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490746    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490750    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490754    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490757    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490760    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490763    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490766    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490769    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490771    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490774    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.494467 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490777    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490780    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490783    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490785    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490788    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490791    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490794    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490797    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490800    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490803    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490805    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490808    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490811    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490814    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490816    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490820    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490823    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490826    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490830    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490833    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.495026 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490835    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490838    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490841    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490844    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490847    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490850    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490852    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490855    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490858    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490860    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490863    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490866    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490869    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490871    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490874    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490877    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490880    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490882    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490885    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.495580 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490888    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490891    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490894    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490896    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490899    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490903    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490906    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490909    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490912    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490915    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490918    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490921    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490924    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490927    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490930    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490933    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490936    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490938    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490941    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.496098 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490944    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490947    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490949    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490952    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490955    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490958    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490960    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490963    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490966    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490971    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490975    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490978    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490982    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490985    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490988    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490991    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490994    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.490997    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491000    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.496606 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491003    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491006    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491022    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491101    2550 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491136    2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491143    2550 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491148    2550 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491154    2550 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491159    2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491166    2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491172    2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491177    2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491182    2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491188    2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491192    2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491195    2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491199    2550 flags.go:64] FLAG: --cgroup-root=""
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491202    2550 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491205    2550 flags.go:64] FLAG: --client-ca-file=""
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491208    2550 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491211    2550 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491217    2550 flags.go:64] FLAG: --cluster-domain=""
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491220    2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 18:00:30.497125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491224    2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491228    2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491231    2550 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491236    2550 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491239    2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491242    2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491246    2550 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491250    2550 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491253    2550 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491256    2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491259    2550 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491265    2550 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491269    2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491272    2550 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491275    2550 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491278    2550 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491281    2550 flags.go:64] FLAG: --enable-server="true"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491285    2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491289    2550 flags.go:64] FLAG: --event-burst="100"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491293    2550 flags.go:64] FLAG: --event-qps="50"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491296    2550 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491299    2550 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491302    2550 flags.go:64] FLAG: --eviction-hard=""
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491306    2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 18:00:30.497737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491309    2550 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491312    2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491315    2550 flags.go:64] FLAG: --eviction-soft=""
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491318    2550 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491322    2550 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491325    2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491330    2550 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491333    2550 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491336    2550 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491339    2550 flags.go:64] FLAG: --feature-gates=""
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491343    2550 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491346    2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491349    2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491352    2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491356    2550 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491359    2550 flags.go:64] FLAG: --help="false"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491362    2550 flags.go:64] FLAG: --hostname-override="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491366    2550 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491370    2550 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491376    2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491380    2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491383    2550 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491386    2550 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 18:00:30.498382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491389    2550 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491393    2550 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491396    2550 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491399    2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491403    2550 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491406    2550 flags.go:64] FLAG: --kube-reserved=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491409    2550 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491412    2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491415    2550 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491418    2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491421    2550 flags.go:64] FLAG: --lock-file=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491424    2550 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491427    2550 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491430    2550 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491436    2550 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491440    2550 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491444    2550 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491447    2550 flags.go:64] FLAG: --logging-format="text"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491450    2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491454    2550 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491457    2550 flags.go:64] FLAG: --manifest-url=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491460    2550 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491465    2550 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491468    2550 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491472    2550 flags.go:64] FLAG: --max-pods="110"
Sep 18 18:00:30.499046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491475    2550 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491479    2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491482    2550 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491487    2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491490    2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491493    2550 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491497    2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491505    2550 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491508    2550 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491512    2550 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491515    2550 flags.go:64] FLAG: --pod-cidr=""
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491519    2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491525    2550 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491528    2550 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491531    2550 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491534    2550 flags.go:64] FLAG: --port="10250"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491538    2550 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491541    2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-054fb3f0f34138f9f"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491545    2550 flags.go:64] FLAG: --qos-reserved=""
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491548    2550 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491551    2550 flags.go:64] FLAG: --register-node="true"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491554    2550 flags.go:64] FLAG: --register-with-taints=""
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491559    2550 flags.go:64] FLAG: --registry-burst="10"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491562    2550 flags.go:64] FLAG: --registry-qps="5"
Sep 18 18:00:30.499722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491565    2550 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491568    2550 flags.go:64] FLAG: --reserved-memory=""
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491572    2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491576    2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491579    2550 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491582    2550 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491585    2550 flags.go:64] FLAG: --runonce="false"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491589    2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491592    2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491595    2550 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491598    2550 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491602    2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491606    2550 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491609    2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491613    2550 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491616    2550 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491619    2550 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491622    2550 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491625    2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491628    2550 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491633    2550 flags.go:64] FLAG: --system-cgroups=""
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491636    2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491641    2550 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491645    2550 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491648    2550 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 18:00:30.500376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491652    2550 flags.go:64] FLAG: --tls-min-version=""
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491655    2550 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491658    2550 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491661    2550 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491664    2550 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491669    2550 flags.go:64] FLAG: --v="2"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491673    2550 flags.go:64] FLAG: --version="false"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491677    2550 flags.go:64] FLAG: --vmodule=""
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491681    2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.491685    2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491804    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491808    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491812    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491815    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491819    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491822    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491825    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491828    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491831    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491836    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491839    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491842    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.501058 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491845    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491847    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491850    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491853    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491856    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491859    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491862    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491865    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491868    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491871    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491874    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491877    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491880    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491883    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491886    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491890    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491893    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491895    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491898    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491901    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.501655 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491904    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491907    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491910    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491913    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491916    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491920    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491924    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491927    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491930    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491935    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491938    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491941    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491943    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491947    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491949    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491952    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491955    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491958    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491961    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491964    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.502335 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491967    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491970    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491973    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491975    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491978    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491983    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491990    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.491996    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492000    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492003    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492006    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492022    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492026    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492029    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492032    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492035    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492038    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492041    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492044    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492047    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.502890 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492049    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492055    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492058    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492061    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492064    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492066    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492069    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492072    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492075    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492078    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492081    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492083    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492087    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492090    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492094    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492096    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492099    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492102    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492105    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.503468 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492109    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492112    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492115    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492118    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492121    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492124    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492127    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.492131    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.492727    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.500205    2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.500221    2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.500242    2550 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.500249    2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500307    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.503987 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500312    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500316    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500320    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500323    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500326    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500329    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500332    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500334    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500337    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500341    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500343    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500347    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500350    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500353    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500356    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500359    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500361    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500364    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500369    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500373    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.504434 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500376    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500380    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500384    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500387    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500390    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500393    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500396    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500399    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500403    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500407    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500410    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500413    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500416    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500419    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500422    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500425    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500428    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500431    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500433    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.504992 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500437    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500439    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500442    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500445    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500448    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500451    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500454    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500457    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500460    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500463    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500466    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500469    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500472    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500475    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500478    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500481    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500484    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500488    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500491    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.505536 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500496    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500499    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500503    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500506    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500509    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500512    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500515    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500518    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500521    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500524    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500527    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500529    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500533    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500536    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500539    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500542    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500544    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500547    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500550    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500553    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.506093 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500556    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500560    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500563    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500566    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500569    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500572    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500574    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500578    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500581    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500584    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500587    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500589    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500592    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500595    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500598    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500601    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500609    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500612    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500615    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.506621 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500617    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.500623    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500763    2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500768    2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500772    2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500775    2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500778    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500781    2550 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500783    2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500786    2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500789    2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500792    2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500795    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500797    2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500800    2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 18:00:30.507147 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500803    2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500807    2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500810    2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500812    2550 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500815    2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500818    2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500821    2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500824    2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500827    2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500830    2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500833    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500835    2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500839    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500841    2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500844    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500850    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500853    2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500856    2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500859    2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500862    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 18:00:30.507595 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500865    2550 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500868    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500870    2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500874    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500877    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500880    2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500882    2550 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500885    2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500888    2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500891    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500894    2550 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500896    2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500899    2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500902    2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500905    2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500908    2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500911    2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500914    2550 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500917    2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500919    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 18:00:30.508130 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500922    2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500925    2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500928    2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500931    2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500934    2550 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500937    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500939    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500942    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500946    2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500948    2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500952    2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500956    2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500960    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500963    2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500966    2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500969    2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500972    2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500975    2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500978    2550 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 18:00:30.508887 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500981    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500984    2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500986    2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500989    2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500992    2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500995    2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.500998    2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501001    2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501004    2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501007    2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501025    2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501029    2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501032    2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501035    2550 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501038    2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501041    2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501044    2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501047    2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501050    2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501052    2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 18:00:30.509704 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501055    2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501058    2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501064    2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501067    2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501071    2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501074    2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:30.501078    2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.501083    2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.501185    2550 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.503298    2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.504427    2550 server.go:1019] "Starting client certificate rotation"
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.504531    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:00:30.510359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.504567    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 18:00:30.510715 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.507183    2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:00:30.510715 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.510600    2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 18:00:30.514541 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.514525    2550 log.go:25] "Validated CRI v1 runtime API"
Sep 18 18:00:30.517576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.517561    2550 log.go:25] "Validated CRI v1 image API"
Sep 18 18:00:30.521219 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.521197    2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 18:00:30.523932 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.523913    2550 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ca98c8c6-1fc7-4474-a56d-7bd5438cc373:/dev/nvme0n1p4 cd12bdff-49c2-4e25-a39b-1a242f1213a1:/dev/nvme0n1p3]
Sep 18 18:00:30.524039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.523931    2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 18:00:30.528101 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.527968    2550 manager.go:216] Machine: {Timestamp:2026-09-18 18:00:30.527481852 +0000 UTC m=+2.214111319 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100461 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f8dafe71c101c2da02fd7480e8346 SystemUUID:ec2f8daf-e71c-101c-2da0-2fd7480e8346 BootID:62d758db-b1f3-48f5-8353-751e25faa5e0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e3:9f:6d:05 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e3:9f:6d:05 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:a1:eb:a9:3e:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 18:00:30.528101 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.528095    2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 18:00:30.528229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.528176    2550 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 18:00:30.529876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.529852    2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 18:00:30.530026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.529877    2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-128.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 18:00:30.531105 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.531094    2550 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 18:00:30.531150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.531108    2550 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 18:00:30.531150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.531116    2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:00:30.531150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.531133    2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 18:00:30.531232 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.531201    2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 18:00:30.536157 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536140    2550 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:00:30.536290 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536279    2550 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 18:00:30.536339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536311    2550 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 18:00:30.536339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536321    2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 18:00:30.536339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536335    2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 18:00:30.536458 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536344    2550 kubelet.go:398] "Adding apiserver pod source"
Sep 18 18:00:30.536458 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536353    2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 18:00:30.536764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536739    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:00:30.536764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.536763    2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 18:00:30.537764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.537743    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:00:30.542337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.539235    2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 18:00:30.544585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.544570    2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 18:00:30.544647 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.544602    2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545842    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545858    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545864    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545870    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545878    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 18:00:30.545891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545883    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545899    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545904    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545911    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545917    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545926    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545932    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545961    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 18:00:30.546111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.545967    2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 18:00:30.549848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.549829    2550 server.go:1268] "Started kubelet"
Sep 18 18:00:30.549942 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.549919    2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 18:00:30.550010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.549966    2550 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 18:00:30.550658 ip-10-0-129-128 systemd[1]: Started Kubernetes Kubelet.
Sep 18 18:00:30.551063 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.550665    2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 18:00:30.551220 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.551135    2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 18:00:30.551426 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.551406    2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-128.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 18:00:30.591135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.591110    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntg76"
Sep 18 18:00:30.591135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.591139    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntg76"
Sep 18 18:00:30.591707 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.591604    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 18:00:30.591707 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.591640    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-128.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 18:00:30.592255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.592095    2550 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 18:00:30.592361 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.592317    2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 18:00:30.597143 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.596102    2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-128.ec2.internal.18d67c2c0c1a39fd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-128.ec2.internal,UID:ip-10-0-129-128.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-128.ec2.internal,},FirstTimestamp:2026-09-18 18:00:30.549801469 +0000 UTC m=+2.236430914,LastTimestamp:2026-09-18 18:00:30.549801469 +0000 UTC m=+2.236430914,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-128.ec2.internal,}"
Sep 18 18:00:30.598352 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.598334    2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 18:00:30.598352 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.598346    2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 18:00:30.598537 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.598478    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:30.598918 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.598904    2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 18:00:30.599035 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.598910    2550 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 18:00:30.599124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599113    2550 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 18:00:30.599264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599247    2550 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 18:00:30.599264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599264    2550 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 18:00:30.599485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599467    2550 factory.go:55] Registering systemd factory
Sep 18 18:00:30.599485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599484    2550 factory.go:223] Registration of the systemd container factory successfully
Sep 18 18:00:30.599774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599761    2550 factory.go:153] Registering CRI-O factory
Sep 18 18:00:30.599825 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599777    2550 factory.go:223] Registration of the crio container factory successfully
Sep 18 18:00:30.599863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599824    2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 18:00:30.599863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599845    2550 factory.go:103] Registering Raw factory
Sep 18 18:00:30.599923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.599864    2550 manager.go:1196] Started watching for new ooms in manager
Sep 18 18:00:30.600469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.600450    2550 manager.go:349] Starting recovery of all containers
Sep 18 18:00:30.610327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.610209    2550 manager.go:354] Recovery completed
Sep 18 18:00:30.610708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.610680    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:30.613396 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.613381    2550 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.614474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.614461    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.616374 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616357    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.616448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616393    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.616448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616410    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.616931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616915    2550 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 18:00:30.616931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616924    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.616931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616930    2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 18:00:30.617121 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.616948    2550 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 18:00:30.617617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.617601    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.617666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.617627    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.617666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.617637    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.619212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.619199    2550 policy_none.go:49] "None policy: Start"
Sep 18 18:00:30.619263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.619215    2550 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 18:00:30.619263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.619226    2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 18:00:30.621283 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.621271    2550 policy_none.go:47] "Start"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.665533    2550 manager.go:337] "Starting Device Plugin manager"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.665571    2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.665584    2550 server.go:87] "Starting device plugin registration server"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.665792    2550 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.665812    2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.665979    2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.666060    2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.666067    2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.667067    2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 18:00:30.672563 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.667117    2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:30.698298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.698273    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 18:00:30.699535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.699516    2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 18:00:30.699612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.699570    2550 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 18:00:30.699612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.699592    2550 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 18:00:30.699681 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.699636    2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 18:00:30.701464 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.701443    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:30.767986 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.767971    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.768695 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.768681    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.768756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.768708    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.768756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.768718    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.768756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.768736    2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.775421 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.775405    2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.775471 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.775424    2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-128.ec2.internal\": node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:30.789499 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.789477    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:30.801088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.800451    2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"]
Sep 18 18:00:30.801144 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.801122    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.801855 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.801842    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.801927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.801862    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.801927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.801873    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.804158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804145    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.804280 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804264    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.804334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804293    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.804842 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804825    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.804910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804849    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.804910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804859    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.804910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804884    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.804910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804905    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.805082 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.804915    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.807131 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.807114    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.807227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.807138    2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 18:00:30.807815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.807801    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 18:00:30.807895 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.807821    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 18:00:30.807895 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.807830    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeHasSufficientPID"
Sep 18 18:00:30.835686 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.835666    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.840273 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.840257    2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-128.ec2.internal\" not found" node="ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.890634 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.890614    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:30.901518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.901498    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.901581 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.901522    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.901581 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:30.901539    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcfdef0994e6b71a1bdabdbc7dab2b8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"5dcfdef0994e6b71a1bdabdbc7dab2b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:30.991610 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:30.991573    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.002751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002726    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.002837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002760    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.002837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002786    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcfdef0994e6b71a1bdabdbc7dab2b8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"5dcfdef0994e6b71a1bdabdbc7dab2b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.002837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002831    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.002837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002834    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5dcfdef0994e6b71a1bdabdbc7dab2b8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-128.ec2.internal\" (UID: \"5dcfdef0994e6b71a1bdabdbc7dab2b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.003016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.002840    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42e2c887edb09aa4d1a711482eb04b8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal\" (UID: \"42e2c887edb09aa4d1a711482eb04b8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.092799 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.092749    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.141889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.141872    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.144153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.144136    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:31.193491 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.193472    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.295915 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.295889    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.398858 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.398808    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.499465 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.499434    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.503962 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.503934    2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 18:00:31.504122 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.504095    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:00:31.504173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.504116    2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 18:00:31.597299 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.597271    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 17:55:30 +0000 UTC" deadline="2028-03-05 17:25:09.86552614 +0000 UTC"
Sep 18 18:00:31.597299 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.597296    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12815h24m38.268233519s"
Sep 18 18:00:31.599516 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.599496    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.700148 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.700128    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.802535 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.802506    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:31.867503 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:31.867470    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcfdef0994e6b71a1bdabdbc7dab2b8.slice/crio-0fd312b9d7bc4612c37d445b526e6b42fac88d7e822d8d7a3c6589faf7759529 WatchSource:0}: Error finding container 0fd312b9d7bc4612c37d445b526e6b42fac88d7e822d8d7a3c6589faf7759529: Status 404 returned error can't find the container with id 0fd312b9d7bc4612c37d445b526e6b42fac88d7e822d8d7a3c6589faf7759529
Sep 18 18:00:31.874056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:31.874042    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:00:31.895748 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:31.895727    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e2c887edb09aa4d1a711482eb04b8e.slice/crio-9d1c290a3df69f71d2356aa57f4897284fb6739b40d7fa845bc87c5ec5217960 WatchSource:0}: Error finding container 9d1c290a3df69f71d2356aa57f4897284fb6739b40d7fa845bc87c5ec5217960: Status 404 returned error can't find the container with id 9d1c290a3df69f71d2356aa57f4897284fb6739b40d7fa845bc87c5ec5217960
Sep 18 18:00:31.903127 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:31.903106    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:32.003154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.003094    2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:32.003282 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.003262    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:32.103454 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.103434    2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-128.ec2.internal\" not found"
Sep 18 18:00:32.139920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.139902    2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:32.203392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.203371    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:32.214966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.214932    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:00:32.216040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.216027    2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal"
Sep 18 18:00:32.225312 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.225294    2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 18:00:32.539085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.539058    2550 apiserver.go:52] "Watching apiserver"
Sep 18 18:00:32.545776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.545755    2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 18:00:32.546249 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.546223    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mqhhp","openshift-cluster-node-tuning-operator/tuned-ztn4s","openshift-multus/multus-additional-cni-plugins-2svdh","openshift-multus/network-metrics-daemon-7bt9q","openshift-network-diagnostics/network-check-target-q85md","openshift-network-operator/iptables-alerter-7vqts","openshift-ovn-kubernetes/ovnkube-node-fxxp8","kube-system/global-pull-secret-syncer-qchwl","kube-system/konnectivity-agent-t9tq7","kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf","openshift-dns/node-resolver-6ndzh","openshift-image-registry/node-ca-rvfwt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal"]
Sep 18 18:00:32.548939 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.548918    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551510    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551597    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551605    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551747    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551914    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pb58m\""
Sep 18 18:00:32.552048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.551966    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.552403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.552169    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 18:00:32.552403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.552196    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 18:00:32.555312 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.555284    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 18:00:32.555421 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.555407    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m8fs6\""
Sep 18 18:00:32.555733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.555708    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.556616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.556594    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.559861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.556604    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.559861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.559846    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nj6s7\""
Sep 18 18:00:32.561415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.560160    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.561415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.560715    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 18:00:32.561415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.560759    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 18:00:32.561415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.561049    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 18:00:32.561415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.561096    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.562221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.561912    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.562221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.561988    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:32.562388 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.562227    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:32.563743 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.562846    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:32.565263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.565242    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.567610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.567588    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.567869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.567850    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 18:00:32.567951 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.567906    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.568040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.567979    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7lg48\""
Sep 18 18:00:32.568118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.568102    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.570260 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.570232    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 18:00:32.570360 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.570319    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 18:00:32.570474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.570454    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.570607 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.570589    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn946\""
Sep 18 18:00:32.572739 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.572574    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9qxwv\""
Sep 18 18:00:32.572739 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.572689    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 18:00:32.573007 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.572990    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.575149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.575128    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p5kz7\""
Sep 18 18:00:32.575325 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.575309    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.575445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.575340    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.576600 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.576315    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.576600 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.576394    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:32.580663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.580644    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.582994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.582971    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dpl9x\""
Sep 18 18:00:32.583118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.583097    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.583213 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.583156    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.583326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.583310    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 18:00:32.584777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.584759    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.587085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.587063    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 18:00:32.587190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.587087    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rm9dq\""
Sep 18 18:00:32.587287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.587269    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 18:00:32.599793 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.599773    2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 18:00:32.601699 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.601680    2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 18:00:32.611929 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.611904    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-sys\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.612050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.611940    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.612050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.611983    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-device-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.612050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612008    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-log-socket\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612039    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612066    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr6ck\" (UniqueName: \"kubernetes.io/projected/40a87f8b-11fa-4221-afaf-f876077d444a-kube-api-access-kr6ck\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612093    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-system-cni-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612120    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612145    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612183    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-cnibin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612211    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-socket-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.612275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612263    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-var-lib-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612289    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a87f8b-11fa-4221-afaf-f876077d444a-host\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612336    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-etc-kubernetes\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612396    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-628lx\" (UniqueName: \"kubernetes.io/projected/28641cac-4605-4c9f-af83-a23a3d2c09a8-kube-api-access-628lx\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612438    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-kubelet\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612481    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612508    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-netd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612569    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-script-lib\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.612658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612650    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c62bb116-2b20-47e5-91d1-9ec328fc1016-iptables-alerter-script\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612685    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c62bb116-2b20-47e5-91d1-9ec328fc1016-host-slash\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612730    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-multus-certs\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612762    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783025bc-6935-4e23-9f5e-13bf0a719c7d-hosts-file\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612788    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-conf-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612818    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-daemon-config\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-ovn\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612864    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f1b9143-08b9-41d7-a2f7-7569c371a3df-agent-certs\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612886    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612908    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-run\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612931    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-dbus\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.612981    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613003    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-slash\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613022    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovn-node-metrics-cert\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613045    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2c59\" (UniqueName: \"kubernetes.io/projected/783025bc-6935-4e23-9f5e-13bf0a719c7d-kube-api-access-m2c59\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613082    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-k8s-cni-cncf-io\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613097    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-registration-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.613212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613115    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a87f8b-11fa-4221-afaf-f876077d444a-serviceca\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613137    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-cni-binary-copy\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613160    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-hostroot\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613184    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-systemd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613207    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-node-log\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613239    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-bin\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613263    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-multus\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613295    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-config\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613318    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f1b9143-08b9-41d7-a2f7-7569c371a3df-konnectivity-ca\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613340    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-kubernetes\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613364    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-conf\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-lib-modules\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613440    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-tmp\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613493    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-systemd\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613521    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-kubelet-config\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613606    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-etc-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613646    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-env-overrides\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.614138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613699    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysconfig\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613744    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmmcv\" (UniqueName: \"kubernetes.io/projected/8c80d17d-80ed-46d2-8cb8-8bba630fb593-kube-api-access-zmmcv\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613774    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-sys-fs\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613841    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-systemd-units\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613864    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-var-lib-kubelet\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613887    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783025bc-6935-4e23-9f5e-13bf0a719c7d-tmp-dir\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613922    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxxsq\" (UniqueName: \"kubernetes.io/projected/9649b91e-5c5d-4b53-8685-05e25dfc10d6-kube-api-access-vxxsq\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613955    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-system-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.613995    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-netns\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614025    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614048    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-tuned\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614072    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xg4b\" (UniqueName: \"kubernetes.io/projected/c62bb116-2b20-47e5-91d1-9ec328fc1016-kube-api-access-7xg4b\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614125    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614150    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-bin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614173    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614203    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-kubelet\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.615060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614225    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-netns\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614268    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8kr9\" (UniqueName: \"kubernetes.io/projected/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-kube-api-access-v8kr9\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614300    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614327    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-modprobe-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614360    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-host\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614417    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cnibin\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614442    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-socket-dir-parent\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614589    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kube-api-access-f68db\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614640    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.616003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.614670    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g7d7\" (UniqueName: \"kubernetes.io/projected/68cc169a-4dc4-4458-85f9-46ffb8565d59-kube-api-access-7g7d7\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.624319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.624294    2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 18:00:32.626627 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.626601    2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:32.637256 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.637236    2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 18:00:32.642440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.642310    2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-t9m4n"
Sep 18 18:00:32.651645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.651626    2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-t9m4n"
Sep 18 18:00:32.707476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.707428    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"42e2c887edb09aa4d1a711482eb04b8e","Type":"ContainerStarted","Data":"9d1c290a3df69f71d2356aa57f4897284fb6739b40d7fa845bc87c5ec5217960"}
Sep 18 18:00:32.709511 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.709485    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" event={"ID":"5dcfdef0994e6b71a1bdabdbc7dab2b8","Type":"ContainerStarted","Data":"0fd312b9d7bc4612c37d445b526e6b42fac88d7e822d8d7a3c6589faf7759529"}
Sep 18 18:00:32.714874 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714850    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2c59\" (UniqueName: \"kubernetes.io/projected/783025bc-6935-4e23-9f5e-13bf0a719c7d-kube-api-access-m2c59\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714878    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-k8s-cni-cncf-io\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714894    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-registration-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714910    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a87f8b-11fa-4221-afaf-f876077d444a-serviceca\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714935    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-cni-binary-copy\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714970    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-hostroot\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.714998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714991    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-systemd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.714996    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-k8s-cni-cncf-io\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715016    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-node-log\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715075    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-node-log\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715138    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-registration-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715213    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-bin\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715252    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-multus\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715280    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-config\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715295    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-hostroot\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.715310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715303    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f1b9143-08b9-41d7-a2f7-7569c371a3df-konnectivity-ca\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715352    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-bin\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715396    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-multus\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715513    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-kubernetes\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715555    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-systemd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715574    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-conf\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715599    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-lib-modules\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715622    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-tmp\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715636    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-kubernetes\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715649    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-systemd\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715676    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-kubelet-config\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715702    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-etc-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715726    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-env-overrides\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715733    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40a87f8b-11fa-4221-afaf-f876077d444a-serviceca\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715750    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysconfig\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.715770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715762    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-lib-modules\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715776    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmcv\" (UniqueName: \"kubernetes.io/projected/8c80d17d-80ed-46d2-8cb8-8bba630fb593-kube-api-access-zmmcv\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715818    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-cni-binary-copy\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715816    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-kubelet-config\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715824    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-sys-fs\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715849    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-conf\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715864    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-systemd\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715885    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-systemd-units\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715909    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-sys-fs\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715914    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5f1b9143-08b9-41d7-a2f7-7569c371a3df-konnectivity-ca\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715927    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysconfig\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715918    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-var-lib-kubelet\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715948    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-config\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715964    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-systemd-units\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.715975    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783025bc-6935-4e23-9f5e-13bf0a719c7d-tmp-dir\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716006    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxxsq\" (UniqueName: \"kubernetes.io/projected/9649b91e-5c5d-4b53-8685-05e25dfc10d6-kube-api-access-vxxsq\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716033    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-var-lib-kubelet\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716055    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-etc-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.716515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716067    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-system-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716082    2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716094    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-netns\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716121    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716148    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-tuned\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716158    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-system-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716174    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xg4b\" (UniqueName: \"kubernetes.io/projected/c62bb116-2b20-47e5-91d1-9ec328fc1016-kube-api-access-7xg4b\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716178    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-netns\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716224    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716247    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-env-overrides\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716249    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-bin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-cni-bin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716300    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716351    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-kubelet\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716359    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783025bc-6935-4e23-9f5e-13bf0a719c7d-tmp-dir\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716388    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-netns\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716417    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8kr9\" (UniqueName: \"kubernetes.io/projected/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-kube-api-access-v8kr9\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716430    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-var-lib-kubelet\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.717390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716438    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-cni-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716441    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716477    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-modprobe-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716483    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-etc-selinux\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716504    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-host\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.716516    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716532    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cnibin\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716579    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-modprobe-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716587    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cnibin\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716600    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-host\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.716612    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:33.216580385 +0000 UTC m=+4.903209831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716625    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-netns\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716647    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-socket-dir-parent\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716679    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kube-api-access-f68db\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716697    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-socket-dir-parent\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716714    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716743    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g7d7\" (UniqueName: \"kubernetes.io/projected/68cc169a-4dc4-4458-85f9-46ffb8565d59-kube-api-access-7g7d7\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.718322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716769    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-sys\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716775    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716826    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-sys\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716888    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-device-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716917    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-log-socket\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716943    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.716992    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr6ck\" (UniqueName: \"kubernetes.io/projected/40a87f8b-11fa-4221-afaf-f876077d444a-kube-api-access-kr6ck\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717004    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-device-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717010    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-log-socket\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717017    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-system-cni-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717064    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-system-cni-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717062    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717062    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717114    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717140    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-cnibin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717181    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.719212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717226    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9649b91e-5c5d-4b53-8685-05e25dfc10d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717239    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-socket-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717263    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-var-lib-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717296    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-cnibin\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.717336    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-socket-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.717386    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:00:33.217369104 +0000 UTC m=+4.903998554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717401    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717407    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a87f8b-11fa-4221-afaf-f876077d444a-host\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717434    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717445    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-var-lib-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717461    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-etc-kubernetes\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717490    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40a87f8b-11fa-4221-afaf-f876077d444a-host\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717500    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-628lx\" (UniqueName: \"kubernetes.io/projected/28641cac-4605-4c9f-af83-a23a3d2c09a8-kube-api-access-628lx\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717533    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-etc-kubernetes\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717564    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-kubelet\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717589    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717617    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-netd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717644    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-script-lib\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717672    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c62bb116-2b20-47e5-91d1-9ec328fc1016-iptables-alerter-script\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717699    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c62bb116-2b20-47e5-91d1-9ec328fc1016-host-slash\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717724    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-multus-certs\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717750    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783025bc-6935-4e23-9f5e-13bf0a719c7d-hosts-file\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717777    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-conf-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717806    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-daemon-config\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717809    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-kubelet\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717832    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-ovn\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717836    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717854    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-openvswitch\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f1b9143-08b9-41d7-a2f7-7569c371a3df-agent-certs\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717913    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783025bc-6935-4e23-9f5e-13bf0a719c7d-hosts-file\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717942    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9649b91e-5c5d-4b53-8685-05e25dfc10d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717948    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717978    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-run\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.720981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718003    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-dbus\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718027    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c62bb116-2b20-47e5-91d1-9ec328fc1016-host-slash\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718044    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718070    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-slash\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718078    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-conf-dir\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.717750    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-cni-netd\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718097    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovn-node-metrics-cert\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718140    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-run\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718184    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/28641cac-4605-4c9f-af83-a23a3d2c09a8-host-run-multus-certs\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718291    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-sysctl-d\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718379    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovnkube-script-lib\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718441    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718489    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581c7670-302a-4aaf-b37a-e594cda3d5cb-dbus\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718534    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-host-slash\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718615    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-run-ovn\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.718771    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/28641cac-4605-4c9f-af83-a23a3d2c09a8-multus-daemon-config\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.721697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.719139    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c62bb116-2b20-47e5-91d1-9ec328fc1016-iptables-alerter-script\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.722691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.722623    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-ovn-node-metrics-cert\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.723869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.723847    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-tmp\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.724077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.724059    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5f1b9143-08b9-41d7-a2f7-7569c371a3df-agent-certs\") pod \"konnectivity-agent-t9tq7\" (UID: \"5f1b9143-08b9-41d7-a2f7-7569c371a3df\") " pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.725528 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.725487    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c80d17d-80ed-46d2-8cb8-8bba630fb593-etc-tuned\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.732502 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.732482    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:32.732599 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.732508    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:32.732599 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.732521    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:32.732599 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:32.732588    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:33.232571519 +0000 UTC m=+4.919200965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:32.736911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.736892    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxxsq\" (UniqueName: \"kubernetes.io/projected/9649b91e-5c5d-4b53-8685-05e25dfc10d6-kube-api-access-vxxsq\") pod \"multus-additional-cni-plugins-2svdh\" (UID: \"9649b91e-5c5d-4b53-8685-05e25dfc10d6\") " pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.737813 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.737204    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2c59\" (UniqueName: \"kubernetes.io/projected/783025bc-6935-4e23-9f5e-13bf0a719c7d-kube-api-access-m2c59\") pod \"node-resolver-6ndzh\" (UID: \"783025bc-6935-4e23-9f5e-13bf0a719c7d\") " pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:32.737813 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.737257    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68db\" (UniqueName: \"kubernetes.io/projected/aab06b1e-b63f-4f45-9ac7-484a6ac7467e-kube-api-access-f68db\") pod \"aws-ebs-csi-driver-node-hzmjf\" (UID: \"aab06b1e-b63f-4f45-9ac7-484a6ac7467e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.738085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.738066    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8kr9\" (UniqueName: \"kubernetes.io/projected/297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0-kube-api-access-v8kr9\") pod \"ovnkube-node-fxxp8\" (UID: \"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.738195 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.738163    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr6ck\" (UniqueName: \"kubernetes.io/projected/40a87f8b-11fa-4221-afaf-f876077d444a-kube-api-access-kr6ck\") pod \"node-ca-rvfwt\" (UID: \"40a87f8b-11fa-4221-afaf-f876077d444a\") " pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.738259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.738201    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g7d7\" (UniqueName: \"kubernetes.io/projected/68cc169a-4dc4-4458-85f9-46ffb8565d59-kube-api-access-7g7d7\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:32.739633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.739611    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmmcv\" (UniqueName: \"kubernetes.io/projected/8c80d17d-80ed-46d2-8cb8-8bba630fb593-kube-api-access-zmmcv\") pod \"tuned-ztn4s\" (UID: \"8c80d17d-80ed-46d2-8cb8-8bba630fb593\") " pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.740031 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.739993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-628lx\" (UniqueName: \"kubernetes.io/projected/28641cac-4605-4c9f-af83-a23a3d2c09a8-kube-api-access-628lx\") pod \"multus-mqhhp\" (UID: \"28641cac-4605-4c9f-af83-a23a3d2c09a8\") " pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.740105 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.740047    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xg4b\" (UniqueName: \"kubernetes.io/projected/c62bb116-2b20-47e5-91d1-9ec328fc1016-kube-api-access-7xg4b\") pod \"iptables-alerter-7vqts\" (UID: \"c62bb116-2b20-47e5-91d1-9ec328fc1016\") " pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.868127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.868068    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:32.877076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.877053    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rvfwt"
Sep 18 18:00:32.888289 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.888224    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2svdh"
Sep 18 18:00:32.898165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.898082    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7vqts"
Sep 18 18:00:32.906043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.906021    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:32.916202 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.916130    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mqhhp"
Sep 18 18:00:32.926774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.926749    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ztn4s"
Sep 18 18:00:32.936659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.936635    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf"
Sep 18 18:00:32.944517 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:32.944500    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6ndzh"
Sep 18 18:00:33.221156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:33.221080    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:33.221156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:33.221128    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:33.221383 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.221255    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:33.221383 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.221273    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:33.221383 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.221333    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:00:34.221314303 +0000 UTC m=+5.907943742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:33.221533 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.221428    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:34.221414666 +0000 UTC m=+5.908044106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:33.322539 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:33.322510    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:33.322720 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.322701    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:33.322776 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.322731    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:33.322776 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.322744    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:33.322913 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:33.322809    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:34.322791746 +0000 UTC m=+6.009421196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:33.655217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:33.655175    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:55:32 +0000 UTC" deadline="2028-06-09 14:08:30.567861674 +0000 UTC"
Sep 18 18:00:33.655217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:33.655211    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15116h7m56.912655802s"
Sep 18 18:00:33.962067 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:33.961990    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297e4ba8_5383_4fc6_b55c_3d1b3d5f01d0.slice/crio-af57b770ebfc0c58343236bac1d9bc681a9c0d2534e3c7c5771632a8ed7d2c71 WatchSource:0}: Error finding container af57b770ebfc0c58343236bac1d9bc681a9c0d2534e3c7c5771632a8ed7d2c71: Status 404 returned error can't find the container with id af57b770ebfc0c58343236bac1d9bc681a9c0d2534e3c7c5771632a8ed7d2c71
Sep 18 18:00:34.024945 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.024919    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62bb116_2b20_47e5_91d1_9ec328fc1016.slice/crio-4394cd37514f390ac9a25767e5d37c8b27768b7fa1851575dec0f2083c38b81a WatchSource:0}: Error finding container 4394cd37514f390ac9a25767e5d37c8b27768b7fa1851575dec0f2083c38b81a: Status 404 returned error can't find the container with id 4394cd37514f390ac9a25767e5d37c8b27768b7fa1851575dec0f2083c38b81a
Sep 18 18:00:34.026680 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.026648    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9649b91e_5c5d_4b53_8685_05e25dfc10d6.slice/crio-6d6fbd3487db13ce2da8fbdb241e18a26e5fd7a65a28817bda2363f24cb6426b WatchSource:0}: Error finding container 6d6fbd3487db13ce2da8fbdb241e18a26e5fd7a65a28817bda2363f24cb6426b: Status 404 returned error can't find the container with id 6d6fbd3487db13ce2da8fbdb241e18a26e5fd7a65a28817bda2363f24cb6426b
Sep 18 18:00:34.028483 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.028449    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1b9143_08b9_41d7_a2f7_7569c371a3df.slice/crio-8fb9092403d092f5675d42c68fdc83726efdfc9a468a688c8d9cebccbdb6123f WatchSource:0}: Error finding container 8fb9092403d092f5675d42c68fdc83726efdfc9a468a688c8d9cebccbdb6123f: Status 404 returned error can't find the container with id 8fb9092403d092f5675d42c68fdc83726efdfc9a468a688c8d9cebccbdb6123f
Sep 18 18:00:34.032822 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.032696    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c80d17d_80ed_46d2_8cb8_8bba630fb593.slice/crio-6ae8118e90e3d288f294610bd8741f45cc4497fd2bfdbf51a513bc50ba8dd071 WatchSource:0}: Error finding container 6ae8118e90e3d288f294610bd8741f45cc4497fd2bfdbf51a513bc50ba8dd071: Status 404 returned error can't find the container with id 6ae8118e90e3d288f294610bd8741f45cc4497fd2bfdbf51a513bc50ba8dd071
Sep 18 18:00:34.033895 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.033871    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a87f8b_11fa_4221_afaf_f876077d444a.slice/crio-5e221a4ef30d7ede9eba32cbada4fc79a3332fde681d8ed713b4dc1fe0f7f967 WatchSource:0}: Error finding container 5e221a4ef30d7ede9eba32cbada4fc79a3332fde681d8ed713b4dc1fe0f7f967: Status 404 returned error can't find the container with id 5e221a4ef30d7ede9eba32cbada4fc79a3332fde681d8ed713b4dc1fe0f7f967
Sep 18 18:00:34.036920 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.036896    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab06b1e_b63f_4f45_9ac7_484a6ac7467e.slice/crio-aa114b63702902ad9eda6951573d84e142f6ef42175f3d2cf96d0ee930ecaf32 WatchSource:0}: Error finding container aa114b63702902ad9eda6951573d84e142f6ef42175f3d2cf96d0ee930ecaf32: Status 404 returned error can't find the container with id aa114b63702902ad9eda6951573d84e142f6ef42175f3d2cf96d0ee930ecaf32
Sep 18 18:00:34.037925 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.037900    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783025bc_6935_4e23_9f5e_13bf0a719c7d.slice/crio-7bb2ade0f7fca19f8fc6d710210f844db094eeb120e8bfca62355b4ebaa8777a WatchSource:0}: Error finding container 7bb2ade0f7fca19f8fc6d710210f844db094eeb120e8bfca62355b4ebaa8777a: Status 404 returned error can't find the container with id 7bb2ade0f7fca19f8fc6d710210f844db094eeb120e8bfca62355b4ebaa8777a
Sep 18 18:00:34.038226 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:00:34.038203    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28641cac_4605_4c9f_af83_a23a3d2c09a8.slice/crio-3dee70a71df8ad24b568b018ee780d77373e8d04f95e7ac7fb4f18ea95967db6 WatchSource:0}: Error finding container 3dee70a71df8ad24b568b018ee780d77373e8d04f95e7ac7fb4f18ea95967db6: Status 404 returned error can't find the container with id 3dee70a71df8ad24b568b018ee780d77373e8d04f95e7ac7fb4f18ea95967db6
Sep 18 18:00:34.229862 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.229610    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:34.229862 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.229764    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:34.230043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.229845    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:34.230043 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.229915    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.229892657 +0000 UTC m=+7.916522104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:34.230043 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.229915    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:34.230043 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.230022    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.230008643 +0000 UTC m=+7.916638077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:34.330317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.330293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:34.330489 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.330452    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:34.330489 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.330485    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:34.330650 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.330499    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:34.330650 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.330554    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:36.330536183 +0000 UTC m=+8.017165636 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:34.655780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.655712    2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 17:55:32 +0000 UTC" deadline="2028-04-05 08:07:07.407718051 +0000 UTC"
Sep 18 18:00:34.655780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.655769    2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13550h6m32.751953449s"
Sep 18 18:00:34.700332 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.700303    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:34.700470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.700314    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:34.700470 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.700423    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:34.700470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.700440    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:34.700646 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.700569    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:34.700698 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:34.700649    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:34.713453 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.713420    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mqhhp" event={"ID":"28641cac-4605-4c9f-af83-a23a3d2c09a8","Type":"ContainerStarted","Data":"3dee70a71df8ad24b568b018ee780d77373e8d04f95e7ac7fb4f18ea95967db6"}
Sep 18 18:00:34.714864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.714835    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6ndzh" event={"ID":"783025bc-6935-4e23-9f5e-13bf0a719c7d","Type":"ContainerStarted","Data":"7bb2ade0f7fca19f8fc6d710210f844db094eeb120e8bfca62355b4ebaa8777a"}
Sep 18 18:00:34.716229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.716205    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf" event={"ID":"aab06b1e-b63f-4f45-9ac7-484a6ac7467e","Type":"ContainerStarted","Data":"aa114b63702902ad9eda6951573d84e142f6ef42175f3d2cf96d0ee930ecaf32"}
Sep 18 18:00:34.717788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.717766    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ztn4s" event={"ID":"8c80d17d-80ed-46d2-8cb8-8bba630fb593","Type":"ContainerStarted","Data":"6ae8118e90e3d288f294610bd8741f45cc4497fd2bfdbf51a513bc50ba8dd071"}
Sep 18 18:00:34.719179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.719154    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerStarted","Data":"6d6fbd3487db13ce2da8fbdb241e18a26e5fd7a65a28817bda2363f24cb6426b"}
Sep 18 18:00:34.720497 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.720476    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9tq7" event={"ID":"5f1b9143-08b9-41d7-a2f7-7569c371a3df","Type":"ContainerStarted","Data":"8fb9092403d092f5675d42c68fdc83726efdfc9a468a688c8d9cebccbdb6123f"}
Sep 18 18:00:34.721825 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.721793    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"af57b770ebfc0c58343236bac1d9bc681a9c0d2534e3c7c5771632a8ed7d2c71"}
Sep 18 18:00:34.724850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.724824    2550 generic.go:356] "Generic (PLEG): container finished" podID="42e2c887edb09aa4d1a711482eb04b8e" containerID="28965ec0dd722290ef0745a20e1bec99d198056ae361672016d25cd407da2ca8" exitCode=0
Sep 18 18:00:34.725304 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.725122    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"42e2c887edb09aa4d1a711482eb04b8e","Type":"ContainerDied","Data":"28965ec0dd722290ef0745a20e1bec99d198056ae361672016d25cd407da2ca8"}
Sep 18 18:00:34.727396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.727365    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" event={"ID":"5dcfdef0994e6b71a1bdabdbc7dab2b8","Type":"ContainerStarted","Data":"0f18cbfe2efee96ccb054f923373df9dc7c79b06af1691980a2ef36330ba7bdb"}
Sep 18 18:00:34.729089 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.729065    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvfwt" event={"ID":"40a87f8b-11fa-4221-afaf-f876077d444a","Type":"ContainerStarted","Data":"5e221a4ef30d7ede9eba32cbada4fc79a3332fde681d8ed713b4dc1fe0f7f967"}
Sep 18 18:00:34.730343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.730320    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7vqts" event={"ID":"c62bb116-2b20-47e5-91d1-9ec328fc1016","Type":"ContainerStarted","Data":"4394cd37514f390ac9a25767e5d37c8b27768b7fa1851575dec0f2083c38b81a"}
Sep 18 18:00:34.753555 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:34.753507    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-128.ec2.internal" podStartSLOduration=2.753489038 podStartE2EDuration="2.753489038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:34.75313236 +0000 UTC m=+6.439761817" watchObservedRunningTime="2026-09-18 18:00:34.753489038 +0000 UTC m=+6.440118498"
Sep 18 18:00:35.742064 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:35.741980    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" event={"ID":"42e2c887edb09aa4d1a711482eb04b8e","Type":"ContainerStarted","Data":"7169cb2da0735dbaab2b10e3b078544f38001da43771304a70afac2f7fb10736"}
Sep 18 18:00:36.248097 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.248061    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:36.248272 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.248151    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:36.248328 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.248273    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:36.248396 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.248328    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:40.248311689 +0000 UTC m=+11.934941122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:36.248706 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.248687    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:36.248791 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.248739    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:00:40.24872528 +0000 UTC m=+11.935354713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:36.348613 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.348577    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:36.348786 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.348726    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:36.348786 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.348747    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:36.348786 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.348759    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:36.348964 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.348809    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:40.348792521 +0000 UTC m=+12.035421956 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:36.707497 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.707465    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:36.707668 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.707611    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:36.708037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.708013    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:36.708144 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.708121    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:36.708202 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:36.708172    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:36.708443 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:36.708243    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:38.701717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:38.701041    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:38.701717 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:38.701166    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:38.701717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:38.701575    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:38.701717 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:38.701667    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:38.703389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:38.702437    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:38.703389 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:38.702526    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:40.280319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.280283    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:40.280780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.280344    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:40.280780 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.280455    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:40.280780 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.280475    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:40.280780 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.280530    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:48.280508979 +0000 UTC m=+19.967138422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:40.280780 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.280550    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:00:48.280541191 +0000 UTC m=+19.967170625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:40.381392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.381348    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:40.381564 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.381552    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:40.381643 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.381572    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:40.381643 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.381582    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:40.381643 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.381630    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:00:48.381612548 +0000 UTC m=+20.068241992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:40.701162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.701089    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:40.701306 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.701200    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:40.710316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.709953    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:40.710316 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.710111    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:40.710316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:40.710166    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:40.710316 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:40.710263    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:42.702278 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:42.702246    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:42.702717 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:42.702384    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:42.702717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:42.702526    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:42.702717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:42.702547    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:42.702717 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:42.702614    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:42.702717 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:42.702688    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:44.703132 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:44.703102    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:44.703132 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:44.703121    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:44.703132 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:44.703121    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:44.703625 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:44.703196    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:44.703625 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:44.703320    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:44.703625 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:44.703410    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:46.703272 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:46.703219    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:46.703713 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:46.703304    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:46.703713 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:46.703339    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:46.703713 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:46.703328    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:46.703713 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:46.703455    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:46.703713 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:46.703540    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:48.337726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.337688    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:48.338162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.337747    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:48.338162 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.337855    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:48.338162 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.337909    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:01:04.337895403 +0000 UTC m=+36.024524834 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:00:48.338162 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.337856    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:48.338162 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.337959    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:04.337948169 +0000 UTC m=+36.024577613 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:00:48.438291 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.438261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:48.438447 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.438428    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:00:48.438447 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.438447    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:00:48.438535 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.438456    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:48.438535 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.438499    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:04.438487587 +0000 UTC m=+36.125117017 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:00:48.703241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.703175    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:48.703241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.703188    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:48.703426 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.703309    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:48.703426 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.703394    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:48.703521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:48.703450    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:48.703555 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:48.703526    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:50.701175 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:50.701143    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:50.701630 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:50.701259    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:50.701630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:50.701264    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:50.701630 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:50.701344    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:50.701630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:50.701369    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:50.701630 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:50.701434    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:51.783090 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.783057    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mqhhp" event={"ID":"28641cac-4605-4c9f-af83-a23a3d2c09a8","Type":"ContainerStarted","Data":"d356c939d0df1aa1c5cd15320be70550d0688790b211e26dc38353504387d1e0"}
Sep 18 18:00:51.784519 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.784497    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6ndzh" event={"ID":"783025bc-6935-4e23-9f5e-13bf0a719c7d","Type":"ContainerStarted","Data":"d00d8101b3dfa481dc55eb6c83dc74de549119168cd255aa8f5de0cda7fea66c"}
Sep 18 18:00:51.785855 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.785834    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf" event={"ID":"aab06b1e-b63f-4f45-9ac7-484a6ac7467e","Type":"ContainerStarted","Data":"1142ba3476448a9068dd244d8a5cdcbc9fb6f2b0fec379cae6a8dc6834f7eee5"}
Sep 18 18:00:51.790142 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.790119    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ztn4s" event={"ID":"8c80d17d-80ed-46d2-8cb8-8bba630fb593","Type":"ContainerStarted","Data":"db5075ff29c1ab7b0a2f1d4ebd8774453bee9a4e15710eaea00b46b226c01ab6"}
Sep 18 18:00:51.791450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.791431    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerStarted","Data":"817713ca3e3523f42cfe62a95471ebdbed1408d354a2c31d2df65349f3572fdd"}
Sep 18 18:00:51.792729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.792706    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t9tq7" event={"ID":"5f1b9143-08b9-41d7-a2f7-7569c371a3df","Type":"ContainerStarted","Data":"f36416a38962fed375e689c5c0e234fddcbdd68fbe98fe1c82ee4301ec6c3497"}
Sep 18 18:00:51.794839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.794814    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"41a81bcb6660ad9129bfe58fc8cb7407dceff6705b034524f8a5d7aa289bbe61"}
Sep 18 18:00:51.796212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.796192    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rvfwt" event={"ID":"40a87f8b-11fa-4221-afaf-f876077d444a","Type":"ContainerStarted","Data":"b831389d41e08280d34b13d3ee34a35df5fef0bb4159d56a28df415e3cb0c433"}
Sep 18 18:00:51.806522 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.806486    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-128.ec2.internal" podStartSLOduration=19.806475213 podStartE2EDuration="19.806475213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:35.756176278 +0000 UTC m=+7.442805731" watchObservedRunningTime="2026-09-18 18:00:51.806475213 +0000 UTC m=+23.493104664"
Sep 18 18:00:51.806606 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.806547    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mqhhp" podStartSLOduration=3.424606852 podStartE2EDuration="20.806542906s" totalImagesPullingTime="17.381936054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:51.80620427 +0000 UTC m=+23.492833744" watchObservedRunningTime="2026-09-18 18:00:51.806542906 +0000 UTC m=+23.493172357"
Sep 18 18:00:51.823331 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.823297    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rvfwt" podStartSLOduration=3.478253912 podStartE2EDuration="20.823286787s" totalImagesPullingTime="17.345032875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:51.823003462 +0000 UTC m=+23.509632932" watchObservedRunningTime="2026-09-18 18:00:51.823286787 +0000 UTC m=+23.509916280"
Sep 18 18:00:51.882631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.882579    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6ndzh" podStartSLOduration=3.541499349 podStartE2EDuration="20.882561199s" totalImagesPullingTime="17.34106185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:51.882469617 +0000 UTC m=+23.569099094" watchObservedRunningTime="2026-09-18 18:00:51.882561199 +0000 UTC m=+23.569190652"
Sep 18 18:00:51.882842 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.882808    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ztn4s" podStartSLOduration=3.536197021 podStartE2EDuration="20.882801642s" totalImagesPullingTime="17.346604621s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:51.86656477 +0000 UTC m=+23.553194212" watchObservedRunningTime="2026-09-18 18:00:51.882801642 +0000 UTC m=+23.569431096"
Sep 18 18:00:51.898456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:51.898419    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t9tq7" podStartSLOduration=3.546762182 podStartE2EDuration="20.89840521s" totalImagesPullingTime="17.351643028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:51.898338948 +0000 UTC m=+23.584968424" watchObservedRunningTime="2026-09-18 18:00:51.89840521 +0000 UTC m=+23.585034662"
Sep 18 18:00:52.700234 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.700002    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:52.700380 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.700005    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:52.700380 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.700052    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:52.700513 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:52.700390    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:52.700513 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:52.700444    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:52.700513 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:52.700274    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:52.771182 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.771159    2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 18:00:52.802571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.802542    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf" event={"ID":"aab06b1e-b63f-4f45-9ac7-484a6ac7467e","Type":"ContainerStarted","Data":"c5797f4a74213655daaa7106b99356dea35c0270a223ac789a2ee873c2dbf3b7"}
Sep 18 18:00:52.803951 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.803913    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="817713ca3e3523f42cfe62a95471ebdbed1408d354a2c31d2df65349f3572fdd" exitCode=0
Sep 18 18:00:52.804046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.803991    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"817713ca3e3523f42cfe62a95471ebdbed1408d354a2c31d2df65349f3572fdd"}
Sep 18 18:00:52.806963 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.806949    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:00:52.807282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807264    2550 generic.go:356] "Generic (PLEG): container finished" podID="297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0" containerID="a97d4e0bd9063e3e7fd7313ec84209e762ec97d0437c11504799dc7aa6d64015" exitCode=1
Sep 18 18:00:52.807348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807327    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerDied","Data":"a97d4e0bd9063e3e7fd7313ec84209e762ec97d0437c11504799dc7aa6d64015"}
Sep 18 18:00:52.807393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807373    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"ca5004b75de829f7565652c22edd98886d0244b17f715d2edec06fafb2169815"}
Sep 18 18:00:52.807448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807403    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"ad3aab69625ecb99cb5f45afdb775258f8e5e740c076a8d1db1a334268fa978c"}
Sep 18 18:00:52.807448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807440    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"ec90355a19692d8e4d641497107d0b939cb6349570b03900fe0c6b09ca859e39"}
Sep 18 18:00:52.807511 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.807467    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"656498137cb88980c18e2acdc91ec6a370f17797ada63ca9934a605f3ebbc675"}
Sep 18 18:00:52.808869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.808844    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7vqts" event={"ID":"c62bb116-2b20-47e5-91d1-9ec328fc1016","Type":"ContainerStarted","Data":"05a15e0f09951d07d115ef7129c358c89a2f5c49947873d4b089ae0d07e0f355"}
Sep 18 18:00:52.837722 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:52.837681    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7vqts" podStartSLOduration=4.480534078 podStartE2EDuration="21.837669728s" totalImagesPullingTime="17.35713565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:52.837517607 +0000 UTC m=+24.524147071" watchObservedRunningTime="2026-09-18 18:00:52.837669728 +0000 UTC m=+24.524299180"
Sep 18 18:00:53.677257 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:53.677168    2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T18:00:52.771177195Z","UUID":"b3b78e57-0800-4b54-a3fc-ea75785c6c3d","Handler":null,"Name":"","Endpoint":""}
Sep 18 18:00:53.678576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:53.678557    2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 18:00:53.678682 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:53.678584    2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 18:00:54.703366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.703337    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:54.703366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.703369    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:54.703918 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:54.703456    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:54.703918 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.703515    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:54.703918 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:54.703633    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:54.703918 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:54.703704    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:54.795791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.795765    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:54.796353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.796326    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:54.814480 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.814452    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf" event={"ID":"aab06b1e-b63f-4f45-9ac7-484a6ac7467e","Type":"ContainerStarted","Data":"abc0ca86926a7b8e269b5fe900a4cb3e713f11617849a83bbb77434c5f52ad45"}
Sep 18 18:00:54.818843 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.818817    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:00:54.819666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.819640    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"125487f4959759ffcff76f6e56b2b7789a23bfd7c8457c63c6816bc0a247e4de"}
Sep 18 18:00:54.820816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.819987    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:54.820816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.820377    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t9tq7"
Sep 18 18:00:54.832978 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:54.832916    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hzmjf" podStartSLOduration=4.324941857 podStartE2EDuration="23.832901004s" totalImagesPullingTime="19.507959147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:54.832863744 +0000 UTC m=+26.519493219" watchObservedRunningTime="2026-09-18 18:00:54.832901004 +0000 UTC m=+26.519530482"
Sep 18 18:00:56.700551 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:56.700308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:56.700982 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:56.700616    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:56.700982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:56.700327    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:56.700982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:56.700308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:56.700982 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:56.700711    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:56.700982 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:56.700787    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:58.700521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.700490    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:58.700521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.700506    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:58.701347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.700506    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:58.701347 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:58.700584    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:58.701347 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:58.700661    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:58.701347 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:58.700740    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:58.830375 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.830345    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="089f62b2260f884e8e691c53a8c183969e0654b96cd9448938254769883047b2" exitCode=0
Sep 18 18:00:58.830498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.830438    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"089f62b2260f884e8e691c53a8c183969e0654b96cd9448938254769883047b2"}
Sep 18 18:00:58.835702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.835670    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:00:58.838194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.838167    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"0088200a3edc1f6ed419669ed2278aee38be99e44aa8df9e37498c08bc436f42"}
Sep 18 18:00:58.838541 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.838523    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:58.838611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.838580    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:58.838704 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.838684    2550 scope.go:117] "RemoveContainer" containerID="a97d4e0bd9063e3e7fd7313ec84209e762ec97d0437c11504799dc7aa6d64015"
Sep 18 18:00:58.855263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:58.855234    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:59.697090 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.696764    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qchwl"]
Sep 18 18:00:59.697221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.697111    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:00:59.697279 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:59.697247    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:00:59.698160 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.697588    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7bt9q"]
Sep 18 18:00:59.698160 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.697681    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:00:59.698160 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:59.697786    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:00:59.698372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.698207    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q85md"]
Sep 18 18:00:59.698372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.698296    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:00:59.698464 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:00:59.698403    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:00:59.842458 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.842436    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="8d05a6a20e44b4d10ff3365122da5bc155636c71a9ffec5d2a1e9ad927252225" exitCode=0
Sep 18 18:00:59.842774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.842519    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"8d05a6a20e44b4d10ff3365122da5bc155636c71a9ffec5d2a1e9ad927252225"}
Sep 18 18:00:59.846729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.846711    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:00:59.847091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.847072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" event={"ID":"297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0","Type":"ContainerStarted","Data":"7fd1ba1bd39de3bf99ee6932f785d4779efdb74ebffe7ef8cb9c39cbc12ee156"}
Sep 18 18:00:59.847382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.847362    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:59.861470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.861452    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:00:59.892751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:00:59.892709    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8" podStartSLOduration=11.492444062 podStartE2EDuration="28.892698532s" totalImagesPullingTime="17.40025447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:00:59.891638524 +0000 UTC m=+31.578267987" watchObservedRunningTime="2026-09-18 18:00:59.892698532 +0000 UTC m=+31.579327985"
Sep 18 18:01:00.851193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:00.851162    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="bb69a66f894986b3e28546dd825c5a491c79b76fb992d1441094789b35c337b7" exitCode=0
Sep 18 18:01:00.851492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:00.851239    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"bb69a66f894986b3e28546dd825c5a491c79b76fb992d1441094789b35c337b7"}
Sep 18 18:01:01.700190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:01.700135    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:01.700190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:01.700136    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:01.700190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:01.700152    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:01.700394 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:01.700266    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:01:01.700394 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:01.700327    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:01:01.700457 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:01.700421    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:01:03.700579 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:03.700413    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:03.700933 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:03.700441    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:03.700933 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:03.700458    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:03.700933 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:03.700730    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qchwl" podUID="581c7670-302a-4aaf-b37a-e594cda3d5cb"
Sep 18 18:01:03.700933 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:03.700807    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7bt9q" podUID="68cc169a-4dc4-4458-85f9-46ffb8565d59"
Sep 18 18:01:03.700933 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:03.700644    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q85md" podUID="8b4e0de8-fba1-4c15-9a26-bd61a9503290"
Sep 18 18:01:04.361985 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.361953    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:04.362221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.362034    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:04.362221 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.362122    2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 18:01:04.362221 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.362132    2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:01:04.362221 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.362198    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret podName:581c7670-302a-4aaf-b37a-e594cda3d5cb nodeName:}" failed. No retries permitted until 2026-09-18 18:01:36.362178875 +0000 UTC m=+68.048808312 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret") pod "global-pull-secret-syncer-qchwl" (UID: "581c7670-302a-4aaf-b37a-e594cda3d5cb") : object "kube-system"/"original-pull-secret" not registered
Sep 18 18:01:04.362221 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.362221    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs podName:68cc169a-4dc4-4458-85f9-46ffb8565d59 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:36.362210094 +0000 UTC m=+68.048839531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs") pod "network-metrics-daemon-7bt9q" (UID: "68cc169a-4dc4-4458-85f9-46ffb8565d59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 18:01:04.462991 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.462956    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:04.463155 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.463129    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 18:01:04.463155 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.463156    2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 18:01:04.463297 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.463169    2550 projected.go:196] Error preparing data for projected volume kube-api-access-ccrdd for pod openshift-network-diagnostics/network-check-target-q85md: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:01:04.463297 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.463227    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd podName:8b4e0de8-fba1-4c15-9a26-bd61a9503290 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:36.463209095 +0000 UTC m=+68.149838531 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ccrdd" (UniqueName: "kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd") pod "network-check-target-q85md" (UID: "8b4e0de8-fba1-4c15-9a26-bd61a9503290") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 18:01:04.553229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.553205    2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-128.ec2.internal" event="NodeReady"
Sep 18 18:01:04.553369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.553330    2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 18:01:04.592376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.592353    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:04.622869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.622846    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bnwbx"]
Sep 18 18:01:04.623042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.622932    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.626140 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.625914    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 18:01:04.626140 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.625993    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzvvm\""
Sep 18 18:01:04.626403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.626383    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 18:01:04.626832 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.626811    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 18:01:04.633051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.632901    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 18:01:04.643263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.643244    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v52xc"]
Sep 18 18:01:04.643387 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.643365    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.646168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.646015    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 18:01:04.646168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.646095    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 18:01:04.646309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.646176    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhsq2\""
Sep 18 18:01:04.663837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.663815    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.663953 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.663817    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:04.664021 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.663972    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v52xc"]
Sep 18 18:01:04.664021 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.663997    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bnwbx"]
Sep 18 18:01:04.664119 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.664027    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cgncr"]
Sep 18 18:01:04.666476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.666456    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wnjwf\""
Sep 18 18:01:04.666764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.666745    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 18:01:04.666764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.666754    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 18:01:04.666908 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.666767    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 18:01:04.683269 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.683253    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cgncr"]
Sep 18 18:01:04.683361 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.683348    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.685381 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.685366    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 18:01:04.685875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.685817    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 18:01:04.685875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.685830    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 18:01:04.685875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.685863    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 18:01:04.686100 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.685973    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dd246\""
Sep 18 18:01:04.765287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765265    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765296    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6033eb70-c797-402e-8355-bb174ecb6106-data-volume\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765322    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-tmp-dir\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765350    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjcnd\" (UniqueName: \"kubernetes.io/projected/6033eb70-c797-402e-8355-bb174ecb6106-kube-api-access-mjcnd\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765423    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-cert\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765448    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765479    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snkkf\" (UniqueName: \"kubernetes.io/projected/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-kube-api-access-snkkf\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765500    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765558    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.765684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765665    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.766145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765703    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.766145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.765730    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.766145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766107    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.766292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766188    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htv2g\" (UniqueName: \"kubernetes.io/projected/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-kube-api-access-htv2g\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.768087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766440    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9qkv\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.768087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766510    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6033eb70-c797-402e-8355-bb174ecb6106-crio-socket\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.768087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766546    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.768087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.766577    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-config-volume\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.867450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867422    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9qkv\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867456    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6033eb70-c797-402e-8355-bb174ecb6106-crio-socket\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867476    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867498    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-config-volume\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867547    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867572    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6033eb70-c797-402e-8355-bb174ecb6106-data-volume\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.867608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867594    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-tmp-dir\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867619    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867646    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjcnd\" (UniqueName: \"kubernetes.io/projected/6033eb70-c797-402e-8355-bb174ecb6106-kube-api-access-mjcnd\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867672    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-cert\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867693    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867713    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snkkf\" (UniqueName: \"kubernetes.io/projected/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-kube-api-access-snkkf\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867734    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867757    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.867752    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6033eb70-c797-402e-8355-bb174ecb6106-crio-socket\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.867840    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:04.867888 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.867858    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:04.868382 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.867918    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.36789713 +0000 UTC m=+37.054526581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:04.868382 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.867953    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls podName:b84f3fac-d7dc-4b4b-ba72-e27f520a0713 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.36792673 +0000 UTC m=+37.054556179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls") pod "dns-default-bnwbx" (UID: "b84f3fac-d7dc-4b4b-ba72-e27f520a0713") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:04.868382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868055    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.868382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868091    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868850    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868897    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868956    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htv2g\" (UniqueName: \"kubernetes.io/projected/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-kube-api-access-htv2g\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868492    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-config-volume\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868654    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6033eb70-c797-402e-8355-bb174ecb6106-data-volume\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.868433    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.869122    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.869132    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f7d546f5-g2mkb: secret "image-registry-tls" not found
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.869178    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.369163252 +0000 UTC m=+37.055792696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.868348    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-tmp-dir\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:04.869201    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls podName:eb59ef17-ce1b-49e9-a00a-273091d4c34c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:05.36918417 +0000 UTC m=+37.055813615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls") pod "image-registry-66f7d546f5-g2mkb" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c") : secret "image-registry-tls" not found
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.869243    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.869583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.869532    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.872385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.872342    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.872486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.872412    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.872486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.872466    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-cert\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.876398 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.876375    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9qkv\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.878033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.877859    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:04.878155 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.878135    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snkkf\" (UniqueName: \"kubernetes.io/projected/1e9ca0b1-e151-4043-bb0b-0c7344d8e10a-kube-api-access-snkkf\") pod \"ingress-canary-v52xc\" (UID: \"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a\") " pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:04.878457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.878417    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htv2g\" (UniqueName: \"kubernetes.io/projected/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-kube-api-access-htv2g\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:04.879435 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.879413    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjcnd\" (UniqueName: \"kubernetes.io/projected/6033eb70-c797-402e-8355-bb174ecb6106-kube-api-access-mjcnd\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:04.972762 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:04.972734    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v52xc"
Sep 18 18:01:05.344170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.344141    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6ndzh_783025bc-6935-4e23-9f5e-13bf0a719c7d/dns-node-resolver/0.log"
Sep 18 18:01:05.373394 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.373362    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:05.373537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.373434    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:05.373537 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373462    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:05.373537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.373481    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:05.373537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.373510    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:05.373537 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373537    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.373519534 +0000 UTC m=+38.060148977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373578    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373586    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373614    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.373603803 +0000 UTC m=+38.060233233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373639    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls podName:b84f3fac-d7dc-4b4b-ba72-e27f520a0713 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.373624029 +0000 UTC m=+38.060253466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls") pod "dns-default-bnwbx" (UID: "b84f3fac-d7dc-4b4b-ba72-e27f520a0713") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373654    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373667    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f7d546f5-g2mkb: secret "image-registry-tls" not found
Sep 18 18:01:05.373783 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:05.373716    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls podName:eb59ef17-ce1b-49e9-a00a-273091d4c34c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:06.373707309 +0000 UTC m=+38.060336744 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls") pod "image-registry-66f7d546f5-g2mkb" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c") : secret "image-registry-tls" not found
Sep 18 18:01:05.700324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.700254    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:05.700324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.700290    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:05.700324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.700296    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:05.702532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.702514    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fqhj6\""
Sep 18 18:01:05.703205 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.703188    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:01:05.703309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.703282    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:01:05.703309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.703298    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:01:05.703406 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.703377    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kxcq\""
Sep 18 18:01:05.703448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.703418    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:01:05.708871 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.708852    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"]
Sep 18 18:01:05.711363 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.711348    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"
Sep 18 18:01:05.713441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.713418    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 18 18:01:05.713532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.713487    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 18 18:01:05.714039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.714023    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-49czc\""
Sep 18 18:01:05.719727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.719708    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"]
Sep 18 18:01:05.878802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.878779    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm5lc\" (UniqueName: \"kubernetes.io/projected/bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35-kube-api-access-sm5lc\") pod \"migrator-76794cbd6c-l7pg6\" (UID: \"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"
Sep 18 18:01:05.979931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.979861    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm5lc\" (UniqueName: \"kubernetes.io/projected/bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35-kube-api-access-sm5lc\") pod \"migrator-76794cbd6c-l7pg6\" (UID: \"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"
Sep 18 18:01:05.991635 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:05.991481    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm5lc\" (UniqueName: \"kubernetes.io/projected/bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35-kube-api-access-sm5lc\") pod \"migrator-76794cbd6c-l7pg6\" (UID: \"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"
Sep 18 18:01:06.033719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.033699    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"
Sep 18 18:01:06.035084 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.035053    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v52xc"]
Sep 18 18:01:06.057873 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:06.057849    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9ca0b1_e151_4043_bb0b_0c7344d8e10a.slice/crio-88b39ebb46c872cadd219e179b45305b6bfeb1ffbb44de72ae12f28831d44751 WatchSource:0}: Error finding container 88b39ebb46c872cadd219e179b45305b6bfeb1ffbb44de72ae12f28831d44751: Status 404 returned error can't find the container with id 88b39ebb46c872cadd219e179b45305b6bfeb1ffbb44de72ae12f28831d44751
Sep 18 18:01:06.303188 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.303166    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6"]
Sep 18 18:01:06.306156 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:06.306119    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0edd7c_0b67_46ff_b5b3_5f7880ca1c35.slice/crio-7de6dd5d473576af0c45f921591cef44d74c466e624922c23a536cf79a898162 WatchSource:0}: Error finding container 7de6dd5d473576af0c45f921591cef44d74c466e624922c23a536cf79a898162: Status 404 returned error can't find the container with id 7de6dd5d473576af0c45f921591cef44d74c466e624922c23a536cf79a898162
Sep 18 18:01:06.382670 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.382641    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:06.382761 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.382691    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:06.382761 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.382733    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:06.382836 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.382767    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:06.382836 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382783    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:06.382905 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382838    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:06.382905 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382852    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls podName:b84f3fac-d7dc-4b4b-ba72-e27f520a0713 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.382831951 +0000 UTC m=+40.069461397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls") pod "dns-default-bnwbx" (UID: "b84f3fac-d7dc-4b4b-ba72-e27f520a0713") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:06.382905 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382862    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:06.382905 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382877    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.382867901 +0000 UTC m=+40.069497346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:06.383070 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382909    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.382897556 +0000 UTC m=+40.069527001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:06.383070 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382865    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:06.383070 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382926    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f7d546f5-g2mkb: secret "image-registry-tls" not found
Sep 18 18:01:06.383070 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:06.382975    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls podName:eb59ef17-ce1b-49e9-a00a-273091d4c34c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:08.382966688 +0000 UTC m=+40.069596122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls") pod "image-registry-66f7d546f5-g2mkb" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c") : secret "image-registry-tls" not found
Sep 18 18:01:06.724078 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.724011    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvfwt_40a87f8b-11fa-4221-afaf-f876077d444a/node-ca/0.log"
Sep 18 18:01:06.866605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.866564    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v52xc" event={"ID":"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a","Type":"ContainerStarted","Data":"88b39ebb46c872cadd219e179b45305b6bfeb1ffbb44de72ae12f28831d44751"}
Sep 18 18:01:06.871764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.871736    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="b423841ab1643dd35cb3039a9d252a700eac764fe627c9070b643e4323c82f55" exitCode=0
Sep 18 18:01:06.871891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.871829    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"b423841ab1643dd35cb3039a9d252a700eac764fe627c9070b643e4323c82f55"}
Sep 18 18:01:06.873984 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:06.873957    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6" event={"ID":"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35","Type":"ContainerStarted","Data":"7de6dd5d473576af0c45f921591cef44d74c466e624922c23a536cf79a898162"}
Sep 18 18:01:07.879755 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:07.879719    2550 generic.go:356] "Generic (PLEG): container finished" podID="9649b91e-5c5d-4b53-8685-05e25dfc10d6" containerID="262ed373bfd3ddbbda9b0704317fd40afbc996f242cd2f75d99a387542942853" exitCode=0
Sep 18 18:01:07.880215 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:07.879778    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerDied","Data":"262ed373bfd3ddbbda9b0704317fd40afbc996f242cd2f75d99a387542942853"}
Sep 18 18:01:08.400721 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.400680    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:08.400906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.400751    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:08.400906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.400826    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:08.400906 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.400850    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:08.400906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.400871    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.400902    2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.400920    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.400899949 +0000 UTC m=+44.087529403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.400976    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.400985    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.400966377 +0000 UTC m=+44.087595829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : configmap "kube-rbac-proxy" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.401001    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.401015    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls podName:b84f3fac-d7dc-4b4b-ba72-e27f520a0713 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.40100323 +0000 UTC m=+44.087632660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls") pod "dns-default-bnwbx" (UID: "b84f3fac-d7dc-4b4b-ba72-e27f520a0713") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.401019    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f7d546f5-g2mkb: secret "image-registry-tls" not found
Sep 18 18:01:08.401154 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:08.401052    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls podName:eb59ef17-ce1b-49e9-a00a-273091d4c34c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:12.401042773 +0000 UTC m=+44.087672207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls") pod "image-registry-66f7d546f5-g2mkb" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c") : secret "image-registry-tls" not found
Sep 18 18:01:08.886001 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.885788    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2svdh" event={"ID":"9649b91e-5c5d-4b53-8685-05e25dfc10d6","Type":"ContainerStarted","Data":"6e68bbf2db4edea2ded6a0c181ca2b1b7cbacc756174b9e2e74ced9c0584f17f"}
Sep 18 18:01:08.887657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.887636    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6" event={"ID":"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35","Type":"ContainerStarted","Data":"b8163a05d357ca1a1cff634d8ded7ecea9530d194c7c5254e55cad7784f77820"}
Sep 18 18:01:08.911730 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:08.911677    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2svdh" podStartSLOduration=8.877715887 podStartE2EDuration="37.911666141s" totalImagesPullingTime="29.033950254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:08.911356195 +0000 UTC m=+40.597985678" watchObservedRunningTime="2026-09-18 18:01:08.911666141 +0000 UTC m=+40.598295590"
Sep 18 18:01:09.892123 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:09.892082    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6" event={"ID":"bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35","Type":"ContainerStarted","Data":"c7c99df05aedc0bd41530201e57c9ac1a27697008f3a58b667b81e6009acd5c6"}
Sep 18 18:01:09.893606 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:09.893583    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v52xc" event={"ID":"1e9ca0b1-e151-4043-bb0b-0c7344d8e10a","Type":"ContainerStarted","Data":"81a6ef29e9831391fcfca8ba58cd3340e2a295aac83b2ea090e5f7eb5f051641"}
Sep 18 18:01:09.908184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:09.908146    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-l7pg6" podStartSLOduration=3.313159674 podStartE2EDuration="4.908133323s" totalImagesPullingTime="1.594973649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:09.907738659 +0000 UTC m=+41.594368132" watchObservedRunningTime="2026-09-18 18:01:09.908133323 +0000 UTC m=+41.594762775"
Sep 18 18:01:09.921918 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:09.921882    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v52xc" podStartSLOduration=2.865159498 podStartE2EDuration="5.921868136s" totalImagesPullingTime="3.056708638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:09.921366636 +0000 UTC m=+41.607996121" watchObservedRunningTime="2026-09-18 18:01:09.921868136 +0000 UTC m=+41.608497587"
Sep 18 18:01:12.433870 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:12.433834    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:12.433880    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:12.433920    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:12.433973    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.433986    2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434044    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls podName:6033eb70-c797-402e-8355-bb174ecb6106 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:20.434028845 +0000 UTC m=+52.120658280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cgncr" (UID: "6033eb70-c797-402e-8355-bb174ecb6106") : secret "insights-runtime-extractor-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434066    2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434085    2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434104    2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66f7d546f5-g2mkb: secret "image-registry-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434121    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls podName:b84f3fac-d7dc-4b4b-ba72-e27f520a0713 nodeName:}" failed. No retries permitted until 2026-09-18 18:01:20.434105337 +0000 UTC m=+52.120734781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls") pod "dns-default-bnwbx" (UID: "b84f3fac-d7dc-4b4b-ba72-e27f520a0713") : secret "dns-default-metrics-tls" not found
Sep 18 18:01:12.434301 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:12.434157    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls podName:eb59ef17-ce1b-49e9-a00a-273091d4c34c nodeName:}" failed. No retries permitted until 2026-09-18 18:01:20.434139715 +0000 UTC m=+52.120769150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls") pod "image-registry-66f7d546f5-g2mkb" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c") : secret "image-registry-tls" not found
Sep 18 18:01:12.434626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:12.434478    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6033eb70-c797-402e-8355-bb174ecb6106-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:20.488699 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.488663    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:20.489199 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.488713    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:20.489199 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.488785    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:20.493112 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.493049    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6033eb70-c797-402e-8355-bb174ecb6106-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cgncr\" (UID: \"6033eb70-c797-402e-8355-bb174ecb6106\") " pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:20.493112 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.493080    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"image-registry-66f7d546f5-g2mkb\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") " pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:20.493112 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.493091    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b84f3fac-d7dc-4b4b-ba72-e27f520a0713-metrics-tls\") pod \"dns-default-bnwbx\" (UID: \"b84f3fac-d7dc-4b4b-ba72-e27f520a0713\") " pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:20.534677 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.534651    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:20.553940 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.553922    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:20.592980 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.592953    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cgncr"
Sep 18 18:01:20.847295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.847270    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:20.850149 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:20.850126    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb59ef17_ce1b_49e9_a00a_273091d4c34c.slice/crio-795fcae5d79e928cb46d07328b93783bfd117787fe75e2c1cd2d7416c8e6d0e0 WatchSource:0}: Error finding container 795fcae5d79e928cb46d07328b93783bfd117787fe75e2c1cd2d7416c8e6d0e0: Status 404 returned error can't find the container with id 795fcae5d79e928cb46d07328b93783bfd117787fe75e2c1cd2d7416c8e6d0e0
Sep 18 18:01:20.861541 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.861521    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bnwbx"]
Sep 18 18:01:20.864038 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:20.864017    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84f3fac_d7dc_4b4b_ba72_e27f520a0713.slice/crio-09ed12b32e77231a930b54d22d8ed2e8510f06b3e42ea12421d9c44e2827e84e WatchSource:0}: Error finding container 09ed12b32e77231a930b54d22d8ed2e8510f06b3e42ea12421d9c44e2827e84e: Status 404 returned error can't find the container with id 09ed12b32e77231a930b54d22d8ed2e8510f06b3e42ea12421d9c44e2827e84e
Sep 18 18:01:20.886925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.886906    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cgncr"]
Sep 18 18:01:20.889861 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:20.889839    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6033eb70_c797_402e_8355_bb174ecb6106.slice/crio-c1447b1c10d5a632420a7a8fba2b83c456964fcbe9375c2b154227608a31c5f0 WatchSource:0}: Error finding container c1447b1c10d5a632420a7a8fba2b83c456964fcbe9375c2b154227608a31c5f0: Status 404 returned error can't find the container with id c1447b1c10d5a632420a7a8fba2b83c456964fcbe9375c2b154227608a31c5f0
Sep 18 18:01:20.920074 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.920048    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" event={"ID":"eb59ef17-ce1b-49e9-a00a-273091d4c34c","Type":"ContainerStarted","Data":"795fcae5d79e928cb46d07328b93783bfd117787fe75e2c1cd2d7416c8e6d0e0"}
Sep 18 18:01:20.921258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.921238    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cgncr" event={"ID":"6033eb70-c797-402e-8355-bb174ecb6106","Type":"ContainerStarted","Data":"c1447b1c10d5a632420a7a8fba2b83c456964fcbe9375c2b154227608a31c5f0"}
Sep 18 18:01:20.922293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:20.922274    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bnwbx" event={"ID":"b84f3fac-d7dc-4b4b-ba72-e27f520a0713","Type":"ContainerStarted","Data":"09ed12b32e77231a930b54d22d8ed2e8510f06b3e42ea12421d9c44e2827e84e"}
Sep 18 18:01:21.930299 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:21.930264    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" event={"ID":"eb59ef17-ce1b-49e9-a00a-273091d4c34c","Type":"ContainerStarted","Data":"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"}
Sep 18 18:01:21.930717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:21.930390    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:21.932081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:21.932050    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cgncr" event={"ID":"6033eb70-c797-402e-8355-bb174ecb6106","Type":"ContainerStarted","Data":"b3fd3570c0e0d5c6421757b18fac6e6ba1b6076114827fe4ed51cb6050118e4d"}
Sep 18 18:01:22.941516 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:22.941479    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cgncr" event={"ID":"6033eb70-c797-402e-8355-bb174ecb6106","Type":"ContainerStarted","Data":"59e8e975416272786f30a8e5c2465ec9ff7bcfdd605bddb0a06fe2900f668145"}
Sep 18 18:01:24.949403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.949371    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bnwbx" event={"ID":"b84f3fac-d7dc-4b4b-ba72-e27f520a0713","Type":"ContainerStarted","Data":"636deca9512173e70ce2a42ab8734c65771a5bb9d79e8c97a4ce61e7f0629189"}
Sep 18 18:01:24.949788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.949433    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bnwbx" event={"ID":"b84f3fac-d7dc-4b4b-ba72-e27f520a0713","Type":"ContainerStarted","Data":"7f414c4c77bdb316db1e77acc253f0071f23a57373d39184dc6ffe4d7d896f6f"}
Sep 18 18:01:24.949788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.949458    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:24.951624 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.951598    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cgncr" event={"ID":"6033eb70-c797-402e-8355-bb174ecb6106","Type":"ContainerStarted","Data":"15f6d73939a6a455b20c0c5ae3c888662906a90f4a9a3914e870511406065a7e"}
Sep 18 18:01:24.966193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.966153    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" podStartSLOduration=58.966138844 podStartE2EDuration="58.966138844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:21.950842436 +0000 UTC m=+53.637471897" watchObservedRunningTime="2026-09-18 18:01:24.966138844 +0000 UTC m=+56.652768297"
Sep 18 18:01:24.966310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.966244    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bnwbx" podStartSLOduration=17.886864316 podStartE2EDuration="20.966238424s" totalImagesPullingTime="3.079374108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:24.96486664 +0000 UTC m=+56.651496107" watchObservedRunningTime="2026-09-18 18:01:24.966238424 +0000 UTC m=+56.652867876"
Sep 18 18:01:24.980947 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:24.980900    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cgncr" podStartSLOduration=17.859714271 podStartE2EDuration="20.980884189s" totalImagesPullingTime="3.121169918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:01:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:24.980061594 +0000 UTC m=+56.666691067" watchObservedRunningTime="2026-09-18 18:01:24.980884189 +0000 UTC m=+56.667513642"
Sep 18 18:01:27.656704 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.656675    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"]
Sep 18 18:01:27.660106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.660069    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.662815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.662792    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 18:01:27.663265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.663250    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 18:01:27.663711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.663692    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 18:01:27.663804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.663781    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 18 18:01:27.679102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.679079    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"]
Sep 18 18:01:27.698560 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.698540    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:27.707759 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.707734    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-fbhkj"]
Sep 18 18:01:27.709971 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.709931    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:27.715508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.715489    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 18:01:27.715606 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.715569    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 18:01:27.716198 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.716177    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8pb7q\""
Sep 18 18:01:27.726297 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.726280    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-fbhkj"]
Sep 18 18:01:27.737766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.737736    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b748bd-0069-462b-b687-499c3c69623a-tmp\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.737841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.737776    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69b748bd-0069-462b-b687-499c3c69623a-klusterlet-config\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.737841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.737795    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmsmb\" (UniqueName: \"kubernetes.io/projected/69b748bd-0069-462b-b687-499c3c69623a-kube-api-access-vmsmb\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.737841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.737812    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcpnm\" (UniqueName: \"kubernetes.io/projected/73ac416f-3856-4fdb-a9a6-fb93aa044abd-kube-api-access-rcpnm\") pod \"downloads-6594f4c75-fbhkj\" (UID: \"73ac416f-3856-4fdb-a9a6-fb93aa044abd\") " pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:27.810817 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.810796    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c4dd8f4c-l92vt"]
Sep 18 18:01:27.812620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.812606    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.833088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.833064    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4dd8f4c-l92vt"]
Sep 18 18:01:27.838328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838300    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-certificates\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838339    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-image-registry-private-configuration\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838390    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-installation-pull-secrets\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838435    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmsmb\" (UniqueName: \"kubernetes.io/projected/69b748bd-0069-462b-b687-499c3c69623a-kube-api-access-vmsmb\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.838474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838454    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-bound-sa-token\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838470    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-ca-trust-extracted\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838599 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838502    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-trusted-ca\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838599 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838524    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nmqf\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-kube-api-access-6nmqf\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838700 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838614    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-tls\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.838700 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838651    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcpnm\" (UniqueName: \"kubernetes.io/projected/73ac416f-3856-4fdb-a9a6-fb93aa044abd-kube-api-access-rcpnm\") pod \"downloads-6594f4c75-fbhkj\" (UID: \"73ac416f-3856-4fdb-a9a6-fb93aa044abd\") " pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:27.838700 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838686    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b748bd-0069-462b-b687-499c3c69623a-tmp\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.838845 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.838710    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69b748bd-0069-462b-b687-499c3c69623a-klusterlet-config\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.839018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.839000    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69b748bd-0069-462b-b687-499c3c69623a-tmp\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.841036 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.841018    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/69b748bd-0069-462b-b687-499c3c69623a-klusterlet-config\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.863899 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.863873    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcpnm\" (UniqueName: \"kubernetes.io/projected/73ac416f-3856-4fdb-a9a6-fb93aa044abd-kube-api-access-rcpnm\") pod \"downloads-6594f4c75-fbhkj\" (UID: \"73ac416f-3856-4fdb-a9a6-fb93aa044abd\") " pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:27.864162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.864145    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmsmb\" (UniqueName: \"kubernetes.io/projected/69b748bd-0069-462b-b687-499c3c69623a-kube-api-access-vmsmb\") pod \"klusterlet-addon-workmgr-76945cd7ff-pdr7w\" (UID: \"69b748bd-0069-462b-b687-499c3c69623a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:27.939779 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939730    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-trusted-ca\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.939779 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939756    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nmqf\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-kube-api-access-6nmqf\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.939779 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939773    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-tls\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939798    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-certificates\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939815    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-image-registry-private-configuration\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939840    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-installation-pull-secrets\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.939995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-bound-sa-token\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.940061    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-ca-trust-extracted\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.940536    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-ca-trust-extracted\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.940680    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-certificates\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.940982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.940928    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-trusted-ca\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.942189 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.942169    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-registry-tls\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.942642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.942626    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-image-registry-private-configuration\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.942773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.942754    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-installation-pull-secrets\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.950047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.950024    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-bound-sa-token\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.950736 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.950719    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nmqf\" (UniqueName: \"kubernetes.io/projected/c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e-kube-api-access-6nmqf\") pod \"image-registry-6c4dd8f4c-l92vt\" (UID: \"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e\") " pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:27.970084 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:27.970062    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:28.018479 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.018235    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:28.120854 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.120830    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:28.240874 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.240803    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"]
Sep 18 18:01:28.244042 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:28.244018    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b748bd_0069_462b_b687_499c3c69623a.slice/crio-a50ec544cf1e03ff1052cfafb520233238f906ebca29f445714709bb56038bf7 WatchSource:0}: Error finding container a50ec544cf1e03ff1052cfafb520233238f906ebca29f445714709bb56038bf7: Status 404 returned error can't find the container with id a50ec544cf1e03ff1052cfafb520233238f906ebca29f445714709bb56038bf7
Sep 18 18:01:28.288968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.288927    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-fbhkj"]
Sep 18 18:01:28.291257 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:28.291236    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ac416f_3856_4fdb_a9a6_fb93aa044abd.slice/crio-0fa480457747e0bcd8cedec386bcaa94a63de608f6ba05bf041faae9351f0b5a WatchSource:0}: Error finding container 0fa480457747e0bcd8cedec386bcaa94a63de608f6ba05bf041faae9351f0b5a: Status 404 returned error can't find the container with id 0fa480457747e0bcd8cedec386bcaa94a63de608f6ba05bf041faae9351f0b5a
Sep 18 18:01:28.393396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.393373    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4dd8f4c-l92vt"]
Sep 18 18:01:28.397394 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:28.397372    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc01cf412_f69a_47b2_a5d2_b4c2f7c33b9e.slice/crio-82d37cbf0a89844f685426d5103087e012faf193b69eba0c1f5ef89c41159ef0 WatchSource:0}: Error finding container 82d37cbf0a89844f685426d5103087e012faf193b69eba0c1f5ef89c41159ef0: Status 404 returned error can't find the container with id 82d37cbf0a89844f685426d5103087e012faf193b69eba0c1f5ef89c41159ef0
Sep 18 18:01:28.966010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.965967    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-fbhkj" event={"ID":"73ac416f-3856-4fdb-a9a6-fb93aa044abd","Type":"ContainerStarted","Data":"0fa480457747e0bcd8cedec386bcaa94a63de608f6ba05bf041faae9351f0b5a"}
Sep 18 18:01:28.967757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.967729    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt" event={"ID":"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e","Type":"ContainerStarted","Data":"66637afb4ed7bbb3ba40120bc04767cb6e1eac5050d5e84767e5970b82a136fc"}
Sep 18 18:01:28.967901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.967773    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt" event={"ID":"c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e","Type":"ContainerStarted","Data":"82d37cbf0a89844f685426d5103087e012faf193b69eba0c1f5ef89c41159ef0"}
Sep 18 18:01:28.967901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.967890    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:28.969305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.969266    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w" event={"ID":"69b748bd-0069-462b-b687-499c3c69623a","Type":"ContainerStarted","Data":"a50ec544cf1e03ff1052cfafb520233238f906ebca29f445714709bb56038bf7"}
Sep 18 18:01:28.992455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:28.992408    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt" podStartSLOduration=1.992388659 podStartE2EDuration="1.992388659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:28.991662671 +0000 UTC m=+60.678292146" watchObservedRunningTime="2026-09-18 18:01:28.992388659 +0000 UTC m=+60.679018111"
Sep 18 18:01:30.549632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.549595    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"]
Sep 18 18:01:30.552060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.552038    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:30.554727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.554702    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 18 18:01:30.554727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.554714    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9ztwj\""
Sep 18 18:01:30.562756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.562734    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"]
Sep 18 18:01:30.663605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.663577    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-bbvws\" (UID: \"b1ab9906-1291-4351-afa2-84b9764270ca\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:30.763993 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:30.763966    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-bbvws\" (UID: \"b1ab9906-1291-4351-afa2-84b9764270ca\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:30.764128 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:30.764103    2550 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 18 18:01:30.764201 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:30.764177    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates podName:b1ab9906-1291-4351-afa2-84b9764270ca nodeName:}" failed. No retries permitted until 2026-09-18 18:01:31.264156084 +0000 UTC m=+62.950785549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates") pod "prometheus-operator-admission-webhook-5c6747dc7b-bbvws" (UID: "b1ab9906-1291-4351-afa2-84b9764270ca") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 18 18:01:31.268048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.267526    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-bbvws\" (UID: \"b1ab9906-1291-4351-afa2-84b9764270ca\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:31.270683 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.270660    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b1ab9906-1291-4351-afa2-84b9764270ca-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-bbvws\" (UID: \"b1ab9906-1291-4351-afa2-84b9764270ca\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:31.460442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.460411    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:31.741171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.741140    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"]
Sep 18 18:01:31.745868 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:31.745834    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ab9906_1291_4351_afa2_84b9764270ca.slice/crio-e480d8c8d421f2945fe6c1c0f4d480723411b8fb177257aa73f25830f1827ddc WatchSource:0}: Error finding container e480d8c8d421f2945fe6c1c0f4d480723411b8fb177257aa73f25830f1827ddc: Status 404 returned error can't find the container with id e480d8c8d421f2945fe6c1c0f4d480723411b8fb177257aa73f25830f1827ddc
Sep 18 18:01:31.865572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.865506    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fxxp8"
Sep 18 18:01:31.944676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.944645    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:01:31.947535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.947516    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950112    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950141    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950165    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ffzz9\""
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950112    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950226    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 18:01:31.950442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.950169    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 18:01:31.958714 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.958692    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:01:31.974412 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974390    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.974518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974427    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.974518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974443    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.974518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974485    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.974630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.974672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.974662    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:31.980997 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:31.980973    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws" event={"ID":"b1ab9906-1291-4351-afa2-84b9764270ca","Type":"ContainerStarted","Data":"e480d8c8d421f2945fe6c1c0f4d480723411b8fb177257aa73f25830f1827ddc"}
Sep 18 18:01:32.075247 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075216    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.075368 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075324    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.075413 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075383    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.075447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075411    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.075447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075436    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.075508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.075459    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.076452 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.076318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.076452 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.076358    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.076452 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.076409    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.079322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.079287    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.080894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.080867    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.084235 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.084211    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd\") pod \"console-58c6f44d6d-j5pfs\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") " pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:32.259684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:32.259652    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:33.327202 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:33.327113    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:01:33.332527 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:33.332498    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7ea551_7b1c_4d5c_a061_c0c7f403c08c.slice/crio-084bda0f9f803356871212af18cf46f51640a345f5f4a9fa7995a5f1939533d3 WatchSource:0}: Error finding container 084bda0f9f803356871212af18cf46f51640a345f5f4a9fa7995a5f1939533d3: Status 404 returned error can't find the container with id 084bda0f9f803356871212af18cf46f51640a345f5f4a9fa7995a5f1939533d3
Sep 18 18:01:33.991091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:33.991059    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c6f44d6d-j5pfs" event={"ID":"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c","Type":"ContainerStarted","Data":"084bda0f9f803356871212af18cf46f51640a345f5f4a9fa7995a5f1939533d3"}
Sep 18 18:01:34.957884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:34.957835    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bnwbx"
Sep 18 18:01:34.999846 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:34.998814    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws" event={"ID":"b1ab9906-1291-4351-afa2-84b9764270ca","Type":"ContainerStarted","Data":"4bcc466860649a6afbc054b7e0187cc06b368070b8ad86e7ade5a20eadc74347"}
Sep 18 18:01:34.999846 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:34.999791    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:35.005926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:35.005884    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws"
Sep 18 18:01:35.034411 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:35.034348    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-bbvws" podStartSLOduration=2.8422450120000002 podStartE2EDuration="5.034332615s" totalImagesPullingTime="2.192087603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:35.016585267 +0000 UTC m=+66.703214721" watchObservedRunningTime="2026-09-18 18:01:35.034332615 +0000 UTC m=+66.720962063"
Sep 18 18:01:36.419670 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.419617    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:36.420180 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.419679    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:36.422141 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.422113    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 18:01:36.422778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.422750    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 18:01:36.433050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.433023    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68cc169a-4dc4-4458-85f9-46ffb8565d59-metrics-certs\") pod \"network-metrics-daemon-7bt9q\" (UID: \"68cc169a-4dc4-4458-85f9-46ffb8565d59\") " pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:36.433158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.433136    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581c7670-302a-4aaf-b37a-e594cda3d5cb-original-pull-secret\") pod \"global-pull-secret-syncer-qchwl\" (UID: \"581c7670-302a-4aaf-b37a-e594cda3d5cb\") " pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:36.520793 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.520757    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:36.523308 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.523222    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 18:01:36.532893 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.532876    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 18:01:36.544165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.544144    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccrdd\" (UniqueName: \"kubernetes.io/projected/8b4e0de8-fba1-4c15-9a26-bd61a9503290-kube-api-access-ccrdd\") pod \"network-check-target-q85md\" (UID: \"8b4e0de8-fba1-4c15-9a26-bd61a9503290\") " pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:36.615239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.615208    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fqhj6\""
Sep 18 18:01:36.620031 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.620001    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qchwl"
Sep 18 18:01:36.624199 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.624179    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:36.629415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.629393    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kxcq\""
Sep 18 18:01:36.638088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:36.638068    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7bt9q"
Sep 18 18:01:37.528959 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.528914    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:01:37.576050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.575977    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:01:37.576050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.576047    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.584265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.584075    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 18:01:37.629608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629549    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629601    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckzjs\" (UniqueName: \"kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629656    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629697    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629726    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629751    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.629820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.629775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.706626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.706592    2550 patch_prober.go:28] interesting pod/image-registry-66f7d546f5-g2mkb container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 18:01:37.706773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.706649    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:01:37.730769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730744    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.730894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730786    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.730894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730814    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.730894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730839    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.731072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730882    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.731072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730952    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.731072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.730991    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckzjs\" (UniqueName: \"kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.731883 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.731632    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.731883 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.731849    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.732024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.731969    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.732096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.732072    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.733789 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.733763    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.734033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.734009    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.739820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.739797    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckzjs\" (UniqueName: \"kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs\") pod \"console-79bf9b4979-7lw9g\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") " pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:37.888628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:37.888554    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:38.454806 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:38.451716    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7bt9q"]
Sep 18 18:01:39.529108 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:39.529075    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cc169a_4dc4_4458_85f9_46ffb8565d59.slice/crio-f780ef861742f03c651b2a807c1f0249fea3125aeee60e79cb2bf29b350e72b6 WatchSource:0}: Error finding container f780ef861742f03c651b2a807c1f0249fea3125aeee60e79cb2bf29b350e72b6: Status 404 returned error can't find the container with id f780ef861742f03c651b2a807c1f0249fea3125aeee60e79cb2bf29b350e72b6
Sep 18 18:01:39.804555 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:39.804474    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qchwl"]
Sep 18 18:01:39.818763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:39.818736    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q85md"]
Sep 18 18:01:39.823217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:39.823197    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:01:40.019500 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.019468    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7bt9q" event={"ID":"68cc169a-4dc4-4458-85f9-46ffb8565d59","Type":"ContainerStarted","Data":"f780ef861742f03c651b2a807c1f0249fea3125aeee60e79cb2bf29b350e72b6"}
Sep 18 18:01:40.021130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.021098    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w" event={"ID":"69b748bd-0069-462b-b687-499c3c69623a","Type":"ContainerStarted","Data":"8e06d43173abebfb7edd6a33b02ab3cfd76236ac50cfdb77bb3dc499eb7e08de"}
Sep 18 18:01:40.021339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.021321    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:40.023054 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.023030    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w"
Sep 18 18:01:40.038649 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.038595    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76945cd7ff-pdr7w" podStartSLOduration=3.104804547 podStartE2EDuration="13.038582826s" totalImagesPullingTime="9.933778279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:40.037704227 +0000 UTC m=+71.724333679" watchObservedRunningTime="2026-09-18 18:01:40.038582826 +0000 UTC m=+71.725212277"
Sep 18 18:01:40.990140 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.990107    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"]
Sep 18 18:01:40.992489 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.992464    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:40.995767 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.995745    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 18:01:40.995985 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.995965    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 18:01:40.996101 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.995968    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 18:01:40.996101 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.996069    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 18:01:40.996642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.996624    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 18:01:40.996737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:40.996713    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-znrp6\""
Sep 18 18:01:41.010223 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.010202    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"]
Sep 18 18:01:41.026262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.026241    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xh7fp"]
Sep 18 18:01:41.028816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.028798    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.031746 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.031730    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9xvzz\""
Sep 18 18:01:41.031846 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.031818    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 18:01:41.032265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.032195    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 18:01:41.032265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.032234    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 18:01:41.058423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.058391    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb6417c9-ff62-4855-9648-4e2056414030-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.058526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.058468    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.058526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.058511    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qpkq\" (UniqueName: \"kubernetes.io/projected/bb6417c9-ff62-4855-9648-4e2056414030-kube-api-access-2qpkq\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.058631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.058553    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.159416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159391    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qpkq\" (UniqueName: \"kubernetes.io/projected/bb6417c9-ff62-4855-9648-4e2056414030-kube-api-access-2qpkq\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.159545 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159431    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-textfile\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159545 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159486    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.159545 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159517    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84nk7\" (UniqueName: \"kubernetes.io/projected/af3a07a2-265b-468a-8612-e40c5471451f-kube-api-access-84nk7\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159604    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-metrics-client-ca\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159679    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-wtmp\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159712    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb6417c9-ff62-4855-9648-4e2056414030-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.159765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-root\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159772    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159810    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-tls\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.159984 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159915    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.160044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.159990    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-sys\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.160100 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.160044    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.160550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.160529    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb6417c9-ff62-4855-9648-4e2056414030-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.162257 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.162232    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.162588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.162568    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb6417c9-ff62-4855-9648-4e2056414030-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.171524 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.171505    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qpkq\" (UniqueName: \"kubernetes.io/projected/bb6417c9-ff62-4855-9648-4e2056414030-kube-api-access-2qpkq\") pod \"openshift-state-metrics-86c5bc79d6-d2v4l\" (UID: \"bb6417c9-ff62-4855-9648-4e2056414030\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.260673 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260646    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.260783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260688    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-textfile\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.260783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260763    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84nk7\" (UniqueName: \"kubernetes.io/projected/af3a07a2-265b-468a-8612-e40c5471451f-kube-api-access-84nk7\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.260892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260800    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-metrics-client-ca\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.260892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260831    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-wtmp\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260956    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-root\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.260979    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-wtmp\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261002    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261018    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-root\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261038    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-tls\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261333 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261121    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-sys\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261333 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261136    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-textfile\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261333 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261197    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af3a07a2-265b-468a-8612-e40c5471451f-sys\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261336    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.261492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.261415    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af3a07a2-265b-468a-8612-e40c5471451f-metrics-client-ca\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.263418 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.263388    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.263711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.263694    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/af3a07a2-265b-468a-8612-e40c5471451f-node-exporter-tls\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.269721 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.269698    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84nk7\" (UniqueName: \"kubernetes.io/projected/af3a07a2-265b-468a-8612-e40c5471451f-kube-api-access-84nk7\") pod \"node-exporter-xh7fp\" (UID: \"af3a07a2-265b-468a-8612-e40c5471451f\") " pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:41.303578 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.303557    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"
Sep 18 18:01:41.338796 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:41.338716    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xh7fp"
Sep 18 18:01:42.048410 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.048276    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:01:42.051290 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.051268    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053822    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053884    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053826    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053951    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053964    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053886    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b5j84\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053976    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 18:01:42.054093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.053885    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 18:01:42.054565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.054344    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 18:01:42.054565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.054347    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 18:01:42.070790 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.070767    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:01:42.169437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169403    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169450    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169505    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169538    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169601    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2ch7\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169658    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169689    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169823    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169878 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169863    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.169925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.169912    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.170018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.170000    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.170082 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.170033    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.170135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.170083    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271218 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271171    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271222    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271257    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271283    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271323    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271341    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271367    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271391    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271409    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271433    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2ch7\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271466    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271488    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.271623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.271518    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.272217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.272192    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.272330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.272253    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.272330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.272309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.275088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.275019    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.275178 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.275144    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.275509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.275484    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.276490 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.276463    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.276798 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.276774    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.276901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.276809    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.277195 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.277124    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.277821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.277719    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.278024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.278000    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.279830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.279809    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2ch7\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7\") pod \"alertmanager-main-0\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:42.362784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:42.362721    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:01:45.776549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:45.776513    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:01:46.773983 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:46.773948    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581c7670_302a_4aaf_b37a_e594cda3d5cb.slice/crio-09cd1f107e963995d81188aae77b6a03821207c33a54ddc6aa42da4815e6ac8e WatchSource:0}: Error finding container 09cd1f107e963995d81188aae77b6a03821207c33a54ddc6aa42da4815e6ac8e: Status 404 returned error can't find the container with id 09cd1f107e963995d81188aae77b6a03821207c33a54ddc6aa42da4815e6ac8e
Sep 18 18:01:46.774649 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:46.774597    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4e0de8_fba1_4c15_9a26_bd61a9503290.slice/crio-173659e1790b0b588dabd6c96291ab15b8f338119f6267e272b9ad645186c2a4 WatchSource:0}: Error finding container 173659e1790b0b588dabd6c96291ab15b8f338119f6267e272b9ad645186c2a4: Status 404 returned error can't find the container with id 173659e1790b0b588dabd6c96291ab15b8f338119f6267e272b9ad645186c2a4
Sep 18 18:01:46.775801 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:46.775769    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8855353f_d814_44ed_8ea2_e6ef49c7dedc.slice/crio-215a39e91e16c9d843a9c6bace4b4625ea0e3c85e840319e89628ba19ef468cc WatchSource:0}: Error finding container 215a39e91e16c9d843a9c6bace4b4625ea0e3c85e840319e89628ba19ef468cc: Status 404 returned error can't find the container with id 215a39e91e16c9d843a9c6bace4b4625ea0e3c85e840319e89628ba19ef468cc
Sep 18 18:01:46.806337 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:46.806308    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf3a07a2_265b_468a_8612_e40c5471451f.slice/crio-a1c57652fa16fd6da3a737c7912fabd13ba9d58ef1fe58c055cedca36793aac9 WatchSource:0}: Error finding container a1c57652fa16fd6da3a737c7912fabd13ba9d58ef1fe58c055cedca36793aac9: Status 404 returned error can't find the container with id a1c57652fa16fd6da3a737c7912fabd13ba9d58ef1fe58c055cedca36793aac9
Sep 18 18:01:47.049961 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.049879    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q85md" event={"ID":"8b4e0de8-fba1-4c15-9a26-bd61a9503290","Type":"ContainerStarted","Data":"173659e1790b0b588dabd6c96291ab15b8f338119f6267e272b9ad645186c2a4"}
Sep 18 18:01:47.051968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.051922    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-fbhkj" event={"ID":"73ac416f-3856-4fdb-a9a6-fb93aa044abd","Type":"ContainerStarted","Data":"23e2467556aa43079482f686bc3efb10d04168192032beaa5fe5d25d73646b24"}
Sep 18 18:01:47.052179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.052154    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:47.053885 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.053858    2550 patch_prober.go:28] interesting pod/downloads-6594f4c75-fbhkj container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused" start-of-body=
Sep 18 18:01:47.054067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.053924    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6594f4c75-fbhkj" podUID="73ac416f-3856-4fdb-a9a6-fb93aa044abd" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.12:8080/\": dial tcp 10.133.0.12:8080: connect: connection refused"
Sep 18 18:01:47.054149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.054094    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c6f44d6d-j5pfs" event={"ID":"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c","Type":"ContainerStarted","Data":"b4c93ce0bf0e6f832199c250ff7a24325d39df7c1c273f725a50fef5039e0a3f"}
Sep 18 18:01:47.056774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.056720    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh7fp" event={"ID":"af3a07a2-265b-468a-8612-e40c5471451f","Type":"ContainerStarted","Data":"a1c57652fa16fd6da3a737c7912fabd13ba9d58ef1fe58c055cedca36793aac9"}
Sep 18 18:01:47.059234 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.059152    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qchwl" event={"ID":"581c7670-302a-4aaf-b37a-e594cda3d5cb","Type":"ContainerStarted","Data":"09cd1f107e963995d81188aae77b6a03821207c33a54ddc6aa42da4815e6ac8e"}
Sep 18 18:01:47.062877 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.062856    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf9b4979-7lw9g" event={"ID":"8855353f-d814-44ed-8ea2-e6ef49c7dedc","Type":"ContainerStarted","Data":"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"}
Sep 18 18:01:47.063025 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.062893    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf9b4979-7lw9g" event={"ID":"8855353f-d814-44ed-8ea2-e6ef49c7dedc","Type":"ContainerStarted","Data":"215a39e91e16c9d843a9c6bace4b4625ea0e3c85e840319e89628ba19ef468cc"}
Sep 18 18:01:47.075043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.074995    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-fbhkj" podStartSLOduration=1.446419568 podStartE2EDuration="20.074977411s" totalImagesPullingTime="18.628557843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:47.07374927 +0000 UTC m=+78.760378746" watchObservedRunningTime="2026-09-18 18:01:47.074977411 +0000 UTC m=+78.761606865"
Sep 18 18:01:47.081807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.081784    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l"]
Sep 18 18:01:47.086489 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:47.086465    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6417c9_ff62_4855_9648_4e2056414030.slice/crio-100ca88dadc653d2ce5061f32236d8a6d9f405af559f04ff129a5974c2ba2167 WatchSource:0}: Error finding container 100ca88dadc653d2ce5061f32236d8a6d9f405af559f04ff129a5974c2ba2167: Status 404 returned error can't find the container with id 100ca88dadc653d2ce5061f32236d8a6d9f405af559f04ff129a5974c2ba2167
Sep 18 18:01:47.091811 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.091788    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:01:47.099387 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:47.099360    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631a4c9f_814e_4f2c_acdc_33445cc2332b.slice/crio-da3a7c819b509f5f8593a06800da739e338c8b3a35a2fe8843851607db454abe WatchSource:0}: Error finding container da3a7c819b509f5f8593a06800da739e338c8b3a35a2fe8843851607db454abe: Status 404 returned error can't find the container with id da3a7c819b509f5f8593a06800da739e338c8b3a35a2fe8843851607db454abe
Sep 18 18:01:47.104252 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.103811    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79bf9b4979-7lw9g" podStartSLOduration=10.103792542 podStartE2EDuration="10.103792542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:47.102266016 +0000 UTC m=+78.788895469" watchObservedRunningTime="2026-09-18 18:01:47.103792542 +0000 UTC m=+78.790421996"
Sep 18 18:01:47.121764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.121669    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58c6f44d6d-j5pfs" podStartSLOduration=2.6458438109999998 podStartE2EDuration="16.121652541s" totalImagesPullingTime="13.47580873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:47.120043892 +0000 UTC m=+78.806673368" watchObservedRunningTime="2026-09-18 18:01:47.121652541 +0000 UTC m=+78.808281993"
Sep 18 18:01:47.706022 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.705764    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:47.889790 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.889747    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:47.890295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.889867    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:47.896643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:47.896409    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:48.071749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.071694    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l" event={"ID":"bb6417c9-ff62-4855-9648-4e2056414030","Type":"ContainerStarted","Data":"eb183115ad4dc829512f885a327f27870a920789d4ad0b07a37eb9f2102af22e"}
Sep 18 18:01:48.071966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.071763    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l" event={"ID":"bb6417c9-ff62-4855-9648-4e2056414030","Type":"ContainerStarted","Data":"96fe01b79cc75b24887aaf678de1fef854f335268ea679d70dab0d195073c605"}
Sep 18 18:01:48.071966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.071792    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l" event={"ID":"bb6417c9-ff62-4855-9648-4e2056414030","Type":"ContainerStarted","Data":"100ca88dadc653d2ce5061f32236d8a6d9f405af559f04ff129a5974c2ba2167"}
Sep 18 18:01:48.075210 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.074985    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"da3a7c819b509f5f8593a06800da739e338c8b3a35a2fe8843851607db454abe"}
Sep 18 18:01:48.079531 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.079443    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-fbhkj"
Sep 18 18:01:48.081087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:48.081047    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:01:49.980718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:49.980199    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c4dd8f4c-l92vt"
Sep 18 18:01:52.260589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:52.260550    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:01:52.718338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:52.718144    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerName="registry" containerID="cri-o://e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6" gracePeriod=30
Sep 18 18:01:53.895559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.895526    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:53.981564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.981512    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:01:53.982250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.982119    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerName="registry"
Sep 18 18:01:53.982250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.982143    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerName="registry"
Sep 18 18:01:53.982250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.982221    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerName="registry"
Sep 18 18:01:53.987857 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.987833    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.987983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.987875    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.987983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.987900    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9qkv\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.987983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.987924    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.988166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.987985    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.988166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.988036    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.988166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.988067    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.988166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.988090    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca\") pod \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\" (UID: \"eb59ef17-ce1b-49e9-a00a-273091d4c34c\") "
Sep 18 18:01:53.988973 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.988682    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:01:53.989312 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.989232    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:01:53.991979 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.991894    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:01:53.993340 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.992732    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:01:53.993783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.993752    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:01:53.995848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.995785    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv" (OuterVolumeSpecName: "kube-api-access-d9qkv") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "kube-api-access-d9qkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:01:53.996687 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:53.996661    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:01:54.000074 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.000043    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "eb59ef17-ce1b-49e9-a00a-273091d4c34c" (UID: "eb59ef17-ce1b-49e9-a00a-273091d4c34c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:01:54.024863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.024779    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:01:54.024954 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.024887    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.089057 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089030    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9qkv\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-kube-api-access-d9qkv\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089057 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089059    2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-bound-sa-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089075    2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/eb59ef17-ce1b-49e9-a00a-273091d4c34c-ca-trust-extracted\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089086    2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-certificates\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089101    2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/eb59ef17-ce1b-49e9-a00a-273091d4c34c-registry-tls\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089112    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/eb59ef17-ce1b-49e9-a00a-273091d4c34c-trusted-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089122    2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-installation-pull-secrets\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.089222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.089135    2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/eb59ef17-ce1b-49e9-a00a-273091d4c34c-image-registry-private-configuration\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:01:54.131287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.130444    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q85md" event={"ID":"8b4e0de8-fba1-4c15-9a26-bd61a9503290","Type":"ContainerStarted","Data":"2723691bfc2b182b1e364fe9e99e57af8c1dd6fc5e6d412b92df9f18b5ec3a86"}
Sep 18 18:01:54.131287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.131231    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:01:54.141138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.141073    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh7fp" event={"ID":"af3a07a2-265b-468a-8612-e40c5471451f","Type":"ContainerStarted","Data":"64dc3ee559eaf0c23e98b339061a3f8bb744c53193a0736a93ee078a22b2415d"}
Sep 18 18:01:54.146572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.145810    2550 generic.go:356] "Generic (PLEG): container finished" podID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" containerID="e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6" exitCode=0
Sep 18 18:01:54.146572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.145887    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" event={"ID":"eb59ef17-ce1b-49e9-a00a-273091d4c34c","Type":"ContainerDied","Data":"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"}
Sep 18 18:01:54.146572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.145927    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb" event={"ID":"eb59ef17-ce1b-49e9-a00a-273091d4c34c","Type":"ContainerDied","Data":"795fcae5d79e928cb46d07328b93783bfd117787fe75e2c1cd2d7416c8e6d0e0"}
Sep 18 18:01:54.146572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.145987    2550 scope.go:117] "RemoveContainer" containerID="e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"
Sep 18 18:01:54.146572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.146122    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66f7d546f5-g2mkb"
Sep 18 18:01:54.159703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.156366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7bt9q" event={"ID":"68cc169a-4dc4-4458-85f9-46ffb8565d59","Type":"ContainerStarted","Data":"242baeffa22819e71ab0e4e6117bb3bd6dc4211c5fbcde0596355f9ff556c84d"}
Sep 18 18:01:54.160660 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.160441    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q85md" podStartSLOduration=76.039086628 podStartE2EDuration="1m23.160422382s" totalImagesPullingTime="7.121335754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:54.147546663 +0000 UTC m=+85.834176155" watchObservedRunningTime="2026-09-18 18:01:54.160422382 +0000 UTC m=+85.847051835"
Sep 18 18:01:54.169020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.168288    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee" exitCode=0
Sep 18 18:01:54.169020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.168325    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee"}
Sep 18 18:01:54.190356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190326    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190366    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190396    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190453    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190513    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190551    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.190688 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.190599    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqjmh\" (UniqueName: \"kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.195998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.195959    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l" podStartSLOduration=7.536160838 podStartE2EDuration="14.195923096s" totalImagesPullingTime="6.659762258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:54.195169691 +0000 UTC m=+85.881799163" watchObservedRunningTime="2026-09-18 18:01:54.195923096 +0000 UTC m=+85.882552549"
Sep 18 18:01:54.201255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.201231    2550 scope.go:117] "RemoveContainer" containerID="e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"
Sep 18 18:01:54.201863 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:01:54.201786    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6\": container with ID starting with e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6 not found: ID does not exist" containerID="e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"
Sep 18 18:01:54.201863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.201823    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6"} err="failed to get container status \"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6\": rpc error: code = NotFound desc = could not find container \"e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6\": container with ID starting with e6d54ac5a659f89c26bf5180ae38e041e7067f5d4ebb36e50e8dc9ad066da1f6 not found: ID does not exist"
Sep 18 18:01:54.235891 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.235870    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:54.239336 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.239315    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66f7d546f5-g2mkb"]
Sep 18 18:01:54.291113 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291077    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.291270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291142    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.291270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291207    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqjmh\" (UniqueName: \"kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.291626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291601    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.291694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291665    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.291751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.291732    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.293238 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.292318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.293238 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.292737    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.293238 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.292951    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.293238 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.292976    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.293526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.293347    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.295034 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.295014    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.297258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.297239    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.299969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.299793    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqjmh\" (UniqueName: \"kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh\") pod \"console-56f9bdbfdc-vgsf5\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") " pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.347377 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.347348    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:01:54.641004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.640971    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:01:54.644858 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:01:54.644826    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacfb6b8_a9e7_4ac3_9d6a_2482c51e9da8.slice/crio-a1937df4266aafca47a6ae74f9508d350e0487a8a2cc12bb717445f0d3cc8370 WatchSource:0}: Error finding container a1937df4266aafca47a6ae74f9508d350e0487a8a2cc12bb717445f0d3cc8370: Status 404 returned error can't find the container with id a1937df4266aafca47a6ae74f9508d350e0487a8a2cc12bb717445f0d3cc8370
Sep 18 18:01:54.706646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:54.706617    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb59ef17-ce1b-49e9-a00a-273091d4c34c" path="/var/lib/kubelet/pods/eb59ef17-ce1b-49e9-a00a-273091d4c34c/volumes"
Sep 18 18:01:55.176163 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.176123    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-d2v4l" event={"ID":"bb6417c9-ff62-4855-9648-4e2056414030","Type":"ContainerStarted","Data":"a60df6485b767214376545aaf8348ea7ba2af53332eb58de69ffa5744f646378"}
Sep 18 18:01:55.178557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.178523    2550 generic.go:356] "Generic (PLEG): container finished" podID="af3a07a2-265b-468a-8612-e40c5471451f" containerID="64dc3ee559eaf0c23e98b339061a3f8bb744c53193a0736a93ee078a22b2415d" exitCode=0
Sep 18 18:01:55.178702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.178612    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh7fp" event={"ID":"af3a07a2-265b-468a-8612-e40c5471451f","Type":"ContainerDied","Data":"64dc3ee559eaf0c23e98b339061a3f8bb744c53193a0736a93ee078a22b2415d"}
Sep 18 18:01:55.181491 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.181460    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qchwl" event={"ID":"581c7670-302a-4aaf-b37a-e594cda3d5cb","Type":"ContainerStarted","Data":"a5ca43f890d726ea39a751011d247443738d1e94c3b16fde0b3e4a4e3de50d26"}
Sep 18 18:01:55.185366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.185343    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7bt9q" event={"ID":"68cc169a-4dc4-4458-85f9-46ffb8565d59","Type":"ContainerStarted","Data":"ddf12335f75f469a5f5765335d1e54d1cb4e8ab22d745c65b041b12643355453"}
Sep 18 18:01:55.187603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.187565    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f9bdbfdc-vgsf5" event={"ID":"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8","Type":"ContainerStarted","Data":"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"}
Sep 18 18:01:55.187730 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.187605    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f9bdbfdc-vgsf5" event={"ID":"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8","Type":"ContainerStarted","Data":"a1937df4266aafca47a6ae74f9508d350e0487a8a2cc12bb717445f0d3cc8370"}
Sep 18 18:01:55.222015 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.221969    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7bt9q" podStartSLOduration=74.756029246 podStartE2EDuration="1m24.221931107s" totalImagesPullingTime="9.465901861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:55.219655772 +0000 UTC m=+86.906285251" watchObservedRunningTime="2026-09-18 18:01:55.221931107 +0000 UTC m=+86.908560560"
Sep 18 18:01:55.240764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.240715    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qchwl" podStartSLOduration=77.105559693 podStartE2EDuration="1m24.240677629s" totalImagesPullingTime="7.135117936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:00:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:55.23932578 +0000 UTC m=+86.925955255" watchObservedRunningTime="2026-09-18 18:01:55.240677629 +0000 UTC m=+86.927307087"
Sep 18 18:01:55.261892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:55.260601    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f9bdbfdc-vgsf5" podStartSLOduration=2.260587 podStartE2EDuration="2.260587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:55.259730393 +0000 UTC m=+86.946359869" watchObservedRunningTime="2026-09-18 18:01:55.260587 +0000 UTC m=+86.947216452"
Sep 18 18:01:56.195144 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:56.195100    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh7fp" event={"ID":"af3a07a2-265b-468a-8612-e40c5471451f","Type":"ContainerStarted","Data":"a21a2718176725f5b5700dec449375a0f429477562491878347c30cda2b64aa8"}
Sep 18 18:01:56.195563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:56.195176    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xh7fp" event={"ID":"af3a07a2-265b-468a-8612-e40c5471451f","Type":"ContainerStarted","Data":"818675d2c6a1a55cd702c3b7f5b8a141f3a676af7e67b39716637b081d43dbbc"}
Sep 18 18:01:56.217364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:56.217310    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xh7fp" podStartSLOduration=13.907850148 podStartE2EDuration="16.217293372s" totalImagesPullingTime="2.194038709s" totalInitContainerRuntime="115.404515ms" isStatefulPod=true podCreationTimestamp="2026-09-18 18:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:56.216275224 +0000 UTC m=+87.902904677" watchObservedRunningTime="2026-09-18 18:01:56.217293372 +0000 UTC m=+87.903922824"
Sep 18 18:01:57.202080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:57.202050    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"}
Sep 18 18:01:57.202437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:57.202135    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"}
Sep 18 18:01:57.202437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:57.202188    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"}
Sep 18 18:01:58.208315 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:58.208276    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"}
Sep 18 18:01:58.208663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:58.208343    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"}
Sep 18 18:01:59.215692 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:59.215662    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerStarted","Data":"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"}
Sep 18 18:01:59.243522 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:01:59.243479    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.690774691 podStartE2EDuration="17.243463917s" totalImagesPullingTime="10.519319454s" totalInitContainerRuntime="33.369772ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:01:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:01:59.24146746 +0000 UTC m=+90.928096901" watchObservedRunningTime="2026-09-18 18:01:59.243463917 +0000 UTC m=+90.930093369"
Sep 18 18:02:04.348016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:04.347977    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:02:04.348586 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:04.348084    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:02:04.353077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:04.353049    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:02:05.244674 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:05.244644    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:02:05.288327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:05.288291    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:02:12.077524 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.077479    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58c6f44d6d-j5pfs" podUID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" containerName="console" containerID="cri-o://b4c93ce0bf0e6f832199c250ff7a24325d39df7c1c273f725a50fef5039e0a3f" gracePeriod=15
Sep 18 18:02:12.269570 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.269541    2550 generic.go:356] "Generic (PLEG): container finished" podID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" containerID="b4c93ce0bf0e6f832199c250ff7a24325d39df7c1c273f725a50fef5039e0a3f" exitCode=0
Sep 18 18:02:12.269739 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.269615    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c6f44d6d-j5pfs" event={"ID":"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c","Type":"ContainerDied","Data":"b4c93ce0bf0e6f832199c250ff7a24325d39df7c1c273f725a50fef5039e0a3f"}
Sep 18 18:02:12.355211 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.355186    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:02:12.425858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.425830    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.425869    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.425901    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.425931    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.426008    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.426036    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert\") pod \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\" (UID: \"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c\") "
Sep 18 18:02:12.426319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.426288    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca" (OuterVolumeSpecName: "service-ca") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:12.426443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.426407    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config" (OuterVolumeSpecName: "console-config") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:12.426515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.426488    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:12.428267 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.428241    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:12.428343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.428277    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:12.428343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.428279    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd" (OuterVolumeSpecName: "kube-api-access-cgnzd") pod "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" (UID: "aa7ea551-7b1c-4d5c-a061-c0c7f403c08c"). InnerVolumeSpecName "kube-api-access-cgnzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:12.527306 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527280    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:12.527306 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527301    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgnzd\" (UniqueName: \"kubernetes.io/projected/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-kube-api-access-cgnzd\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:12.527442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527316    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:12.527442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527326    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:12.527442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527335    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-console-oauth-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:12.527442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:12.527343    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c-oauth-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:13.274896 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:13.274863    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58c6f44d6d-j5pfs" event={"ID":"aa7ea551-7b1c-4d5c-a061-c0c7f403c08c","Type":"ContainerDied","Data":"084bda0f9f803356871212af18cf46f51640a345f5f4a9fa7995a5f1939533d3"}
Sep 18 18:02:13.275326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:13.274913    2550 scope.go:117] "RemoveContainer" containerID="b4c93ce0bf0e6f832199c250ff7a24325d39df7c1c273f725a50fef5039e0a3f"
Sep 18 18:02:13.275326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:13.274875    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58c6f44d6d-j5pfs"
Sep 18 18:02:13.293286 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:13.293266    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:02:13.302257 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:13.302233    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58c6f44d6d-j5pfs"]
Sep 18 18:02:14.705023 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:14.704983    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" path="/var/lib/kubelet/pods/aa7ea551-7b1c-4d5c-a061-c0c7f403c08c/volumes"
Sep 18 18:02:26.198323 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:26.198294    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q85md"
Sep 18 18:02:30.307248 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.307173    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79bf9b4979-7lw9g" podUID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" containerName="console" containerID="cri-o://09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77" gracePeriod=15
Sep 18 18:02:30.583220 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.583161    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:02:30.653099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653065    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653119    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653141    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckzjs\" (UniqueName: \"kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653156    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653185    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653211    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653471 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653230    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert\") pod \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\" (UID: \"8855353f-d814-44ed-8ea2-e6ef49c7dedc\") "
Sep 18 18:02:30.653590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653561    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:30.653658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653590    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config" (OuterVolumeSpecName: "console-config") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:30.653658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653599    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca" (OuterVolumeSpecName: "service-ca") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:30.653732 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.653669    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:02:30.655217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.655194    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:30.655393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.655372    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs" (OuterVolumeSpecName: "kube-api-access-ckzjs") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "kube-api-access-ckzjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:02:30.655393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.655374    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8855353f-d814-44ed-8ea2-e6ef49c7dedc" (UID: "8855353f-d814-44ed-8ea2-e6ef49c7dedc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:02:30.754013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.753995    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754014    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-oauth-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754023    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-oauth-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754032    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8855353f-d814-44ed-8ea2-e6ef49c7dedc-console-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754040    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-trusted-ca-bundle\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754048    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckzjs\" (UniqueName: \"kubernetes.io/projected/8855353f-d814-44ed-8ea2-e6ef49c7dedc-kube-api-access-ckzjs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:30.754196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:30.754056    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8855353f-d814-44ed-8ea2-e6ef49c7dedc-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:02:31.348910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.348881    2550 generic.go:356] "Generic (PLEG): container finished" podID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" containerID="09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77" exitCode=0
Sep 18 18:02:31.349254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.348929    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79bf9b4979-7lw9g"
Sep 18 18:02:31.349254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.348967    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf9b4979-7lw9g" event={"ID":"8855353f-d814-44ed-8ea2-e6ef49c7dedc","Type":"ContainerDied","Data":"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"}
Sep 18 18:02:31.349254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.349015    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79bf9b4979-7lw9g" event={"ID":"8855353f-d814-44ed-8ea2-e6ef49c7dedc","Type":"ContainerDied","Data":"215a39e91e16c9d843a9c6bace4b4625ea0e3c85e840319e89628ba19ef468cc"}
Sep 18 18:02:31.349254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.349038    2550 scope.go:117] "RemoveContainer" containerID="09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"
Sep 18 18:02:31.356417 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.356403    2550 scope.go:117] "RemoveContainer" containerID="09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"
Sep 18 18:02:31.356660 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:02:31.356640    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77\": container with ID starting with 09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77 not found: ID does not exist" containerID="09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"
Sep 18 18:02:31.356709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.356667    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77"} err="failed to get container status \"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77\": rpc error: code = NotFound desc = could not find container \"09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77\": container with ID starting with 09a3d0b34ac83cc0fe420dfb1d7547639b6761ee6530019b3aba9f8ccfef7f77 not found: ID does not exist"
Sep 18 18:02:31.365024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.365002    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:02:31.369463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:31.369441    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79bf9b4979-7lw9g"]
Sep 18 18:02:32.705377 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:02:32.705338    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" path="/var/lib/kubelet/pods/8855353f-d814-44ed-8ea2-e6ef49c7dedc/volumes"
Sep 18 18:03:01.554872 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.554841    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:01.555385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555128    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="alertmanager" containerID="cri-o://daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf" gracePeriod=120
Sep 18 18:03:01.555385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555188    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-metric" containerID="cri-o://fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe" gracePeriod=120
Sep 18 18:03:01.555385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555255    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-web" containerID="cri-o://098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032" gracePeriod=120
Sep 18 18:03:01.555385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555300    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="config-reloader" containerID="cri-o://18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55" gracePeriod=120
Sep 18 18:03:01.555385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555260    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy" containerID="cri-o://eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048" gracePeriod=120
Sep 18 18:03:01.555634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:01.555222    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="prom-label-proxy" containerID="cri-o://483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356" gracePeriod=120
Sep 18 18:03:02.467400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467371    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356" exitCode=0
Sep 18 18:03:02.467400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467392    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe" exitCode=0
Sep 18 18:03:02.467400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467398    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048" exitCode=0
Sep 18 18:03:02.467400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467404    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55" exitCode=0
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467409    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf" exitCode=0
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467440    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"}
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467500    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"}
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467528    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"}
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467554    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"}
Sep 18 18:03:02.467610 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.467581    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"}
Sep 18 18:03:02.794031 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.794003    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:02.872013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.871987    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872022    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872044    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872065    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872179    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872246    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872280    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872307    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872319    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872337    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872349    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872366    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872397    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2ch7\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872436    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872471    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca\") pod \"631a4c9f-814e-4f2c-acdc-33445cc2332b\" (UID: \"631a4c9f-814e-4f2c-acdc-33445cc2332b\") "
Sep 18 18:03:02.872879 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872688    2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.872879 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.872707    2550 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-alertmanager-main-db\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.873161 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.873133    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:03:02.875492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.875464    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out" (OuterVolumeSpecName: "config-out") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:03:02.875799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.875592    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.876259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.876066    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.876259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.876233    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.876460 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.876397    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume" (OuterVolumeSpecName: "config-volume") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.876931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.876901    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.877053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.877000    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:03:02.877812 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.877788    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7" (OuterVolumeSpecName: "kube-api-access-x2ch7") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "kube-api-access-x2ch7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:03:02.879984 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.879959    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.886639 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.886616    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config" (OuterVolumeSpecName: "web-config") pod "631a4c9f-814e-4f2c-acdc-33445cc2332b" (UID: "631a4c9f-814e-4f2c-acdc-33445cc2332b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:03:02.973690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973643    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973662    2550 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-cluster-tls-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973671    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-main-tls\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973681    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973690    2550 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-out\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973699    2550 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973708    2550 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973716    2550 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-tls-assets\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973723    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2ch7\" (UniqueName: \"kubernetes.io/projected/631a4c9f-814e-4f2c-acdc-33445cc2332b-kube-api-access-x2ch7\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973731    2550 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/631a4c9f-814e-4f2c-acdc-33445cc2332b-web-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:02.973884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:02.973739    2550 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631a4c9f-814e-4f2c-acdc-33445cc2332b-metrics-client-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:03:03.478201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.478171    2550 generic.go:356] "Generic (PLEG): container finished" podID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerID="098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032" exitCode=0
Sep 18 18:03:03.478299 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.478239    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"}
Sep 18 18:03:03.478299 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.478254    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.478393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.478316    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"631a4c9f-814e-4f2c-acdc-33445cc2332b","Type":"ContainerDied","Data":"da3a7c819b509f5f8593a06800da739e338c8b3a35a2fe8843851607db454abe"}
Sep 18 18:03:03.478393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.478347    2550 scope.go:117] "RemoveContainer" containerID="483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"
Sep 18 18:03:03.486129 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.486112    2550 scope.go:117] "RemoveContainer" containerID="fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"
Sep 18 18:03:03.493417 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.493402    2550 scope.go:117] "RemoveContainer" containerID="eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"
Sep 18 18:03:03.499625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.499609    2550 scope.go:117] "RemoveContainer" containerID="098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"
Sep 18 18:03:03.503858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.503839    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:03.506397 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.506382    2550 scope.go:117] "RemoveContainer" containerID="18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"
Sep 18 18:03:03.508589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.508571    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:03.513168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.513154    2550 scope.go:117] "RemoveContainer" containerID="daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"
Sep 18 18:03:03.519557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.519541    2550 scope.go:117] "RemoveContainer" containerID="0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee"
Sep 18 18:03:03.525837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.525819    2550 scope.go:117] "RemoveContainer" containerID="483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"
Sep 18 18:03:03.526115 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.526097    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356\": container with ID starting with 483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356 not found: ID does not exist" containerID="483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"
Sep 18 18:03:03.526175 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526123    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356"} err="failed to get container status \"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356\": rpc error: code = NotFound desc = could not find container \"483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356\": container with ID starting with 483c0c62d74b7dd80ddde4d40c68144c8f9a3514b3f3c61e064d3763e4c6a356 not found: ID does not exist"
Sep 18 18:03:03.526175 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526139    2550 scope.go:117] "RemoveContainer" containerID="fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"
Sep 18 18:03:03.526377 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.526360    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe\": container with ID starting with fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe not found: ID does not exist" containerID="fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"
Sep 18 18:03:03.526442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526386    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe"} err="failed to get container status \"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe\": rpc error: code = NotFound desc = could not find container \"fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe\": container with ID starting with fbe1aa3f7ae2c23bf33ad99ac20263435b6c60bf3373aa84007d1e7b99bebafe not found: ID does not exist"
Sep 18 18:03:03.526442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526408    2550 scope.go:117] "RemoveContainer" containerID="eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"
Sep 18 18:03:03.526655 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.526640    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048\": container with ID starting with eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048 not found: ID does not exist" containerID="eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"
Sep 18 18:03:03.526693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526659    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048"} err="failed to get container status \"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048\": rpc error: code = NotFound desc = could not find container \"eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048\": container with ID starting with eb8aa26737dfd1e993574fbf2d6380888645e86d45acc513b1d19e4c0eb94048 not found: ID does not exist"
Sep 18 18:03:03.526693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526678    2550 scope.go:117] "RemoveContainer" containerID="098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"
Sep 18 18:03:03.526888 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.526872    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032\": container with ID starting with 098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032 not found: ID does not exist" containerID="098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"
Sep 18 18:03:03.526968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526894    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032"} err="failed to get container status \"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032\": rpc error: code = NotFound desc = could not find container \"098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032\": container with ID starting with 098eee6879839bdc3fb7866740f902b5c734c289d7a2ae21189aba6ba151a032 not found: ID does not exist"
Sep 18 18:03:03.526968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.526914    2550 scope.go:117] "RemoveContainer" containerID="18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"
Sep 18 18:03:03.527169 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.527153    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55\": container with ID starting with 18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55 not found: ID does not exist" containerID="18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"
Sep 18 18:03:03.527205 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.527173    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55"} err="failed to get container status \"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55\": rpc error: code = NotFound desc = could not find container \"18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55\": container with ID starting with 18b5aa9ea673adfb23ad0735f6c169a0922458d04ff5f95ca9ab5287682ebf55 not found: ID does not exist"
Sep 18 18:03:03.527205 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.527191    2550 scope.go:117] "RemoveContainer" containerID="daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"
Sep 18 18:03:03.527436 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.527420    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf\": container with ID starting with daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf not found: ID does not exist" containerID="daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"
Sep 18 18:03:03.527496 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.527443    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf"} err="failed to get container status \"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf\": rpc error: code = NotFound desc = could not find container \"daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf\": container with ID starting with daf83519117706135d37bad7c70f3a3a080fa9f64fa222867d1972c77be42bcf not found: ID does not exist"
Sep 18 18:03:03.527496 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.527463    2550 scope.go:117] "RemoveContainer" containerID="0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee"
Sep 18 18:03:03.527677 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:03:03.527661    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee\": container with ID starting with 0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee not found: ID does not exist" containerID="0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee"
Sep 18 18:03:03.527714 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.527681    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee"} err="failed to get container status \"0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee\": rpc error: code = NotFound desc = could not find container \"0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee\": container with ID starting with 0ace14db17107ae98136cb7988d36654b013c87791499a297b93f2c8b0dc57ee not found: ID does not exist"
Sep 18 18:03:03.544258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544239    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:03.544506 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544496    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="config-reloader"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544509    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="config-reloader"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544520    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" containerName="console"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544525    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" containerName="console"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544532    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="init-config-reloader"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544538    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="init-config-reloader"
Sep 18 18:03:03.544543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544544    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-web"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544550    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-web"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544555    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544560    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544566    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-metric"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544572    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-metric"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544583    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="prom-label-proxy"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544588    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="prom-label-proxy"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544597    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" containerName="console"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544601    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" containerName="console"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544610    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="alertmanager"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544615    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="alertmanager"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544651    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8855353f-d814-44ed-8ea2-e6ef49c7dedc" containerName="console"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544659    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="alertmanager"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544663    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="config-reloader"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544669    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-web"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544674    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy-metric"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544680    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="prom-label-proxy"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544685    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7ea551-7b1c-4d5c-a061-c0c7f403c08c" containerName="console"
Sep 18 18:03:03.544712 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.544690    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" containerName="kube-rbac-proxy"
Sep 18 18:03:03.548019 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.548003    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.557581 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.557563    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 18:03:03.557688 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.557607    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 18:03:03.558056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.558043    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b5j84\""
Sep 18 18:03:03.560083 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560030    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 18:03:03.560174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560029    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 18:03:03.560174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560032    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 18:03:03.560258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560059    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 18:03:03.560258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560240    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 18:03:03.560388 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.560374    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 18:03:03.564298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.564284    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 18:03:03.575753 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.575737    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:03.677487 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677463    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677531    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-out\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677549    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-web-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677573    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677590    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677608    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-volume\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677652    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677669    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677695    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677711    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677725    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.677955 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.677742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84lkd\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-kube-api-access-84lkd\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779025 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-out\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779104 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-web-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779104 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779049    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779104 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779074    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779116    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-volume\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779144    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779208    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779253    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779278    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779303    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84lkd\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-kube-api-access-84lkd\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779328    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.779641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.779607    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.780183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.780158    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.780285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.780168    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eac7fbe7-acc3-4395-98c6-f7dda09db45a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782246 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.781896    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-out\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782246 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782117    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782566 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782541    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782662 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782590    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-web-config\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782769    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.782841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782783    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-config-volume\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.783018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.782978    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.783291 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.783271    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.784080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.784059    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/eac7fbe7-acc3-4395-98c6-f7dda09db45a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.802217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.802196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84lkd\" (UniqueName: \"kubernetes.io/projected/eac7fbe7-acc3-4395-98c6-f7dda09db45a-kube-api-access-84lkd\") pod \"alertmanager-main-0\" (UID: \"eac7fbe7-acc3-4395-98c6-f7dda09db45a\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:03.856149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:03.856120    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 18:03:04.139468 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:04.139443    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 18:03:04.483614 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:04.483554    2550 generic.go:356] "Generic (PLEG): container finished" podID="eac7fbe7-acc3-4395-98c6-f7dda09db45a" containerID="a451e8a5543d92af2a79b225e75347a4479b0518a3560413117b8f86ae3c29d5" exitCode=0
Sep 18 18:03:04.483718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:04.483643    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerDied","Data":"a451e8a5543d92af2a79b225e75347a4479b0518a3560413117b8f86ae3c29d5"}
Sep 18 18:03:04.483782 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:04.483721    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"2e4b68de9fc44455643345caeee10013da032e2576ff9bd1f4e156ad650c6a87"}
Sep 18 18:03:04.704826 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:04.704798    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="631a4c9f-814e-4f2c-acdc-33445cc2332b" path="/var/lib/kubelet/pods/631a4c9f-814e-4f2c-acdc-33445cc2332b/volumes"
Sep 18 18:03:05.493088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493043    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"0ea4b80e5947dbe53b296494193ef72a1009146137abc966f38137277f1ca3be"}
Sep 18 18:03:05.493538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493153    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"7523c00916c6b33268c681800249b11dddf3539a27451f9865be4070506ddbed"}
Sep 18 18:03:05.493538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493205    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"367103bfe29d5f3e0f1deccb9bcba5c788629519db7eedff09346604d1fd14a9"}
Sep 18 18:03:05.493538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493254    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"bb404fafe129ab0cae73fab9c3859b25bfa577be0cb4c66682a9af20e10b5eb4"}
Sep 18 18:03:05.493538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493295    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"2cc5673f84061446052634e81dc3b10962cb947dba32d88bb82382cef456f67b"}
Sep 18 18:03:05.493538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.493336    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"eac7fbe7-acc3-4395-98c6-f7dda09db45a","Type":"ContainerStarted","Data":"3784580328f93ba8fa6c5aa824cc334070da9f8282e8ae7938e690dcafe1be63"}
Sep 18 18:03:05.527391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:05.527338    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.513265483 podStartE2EDuration="2.527320504s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.055021ms" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:05.524735095 +0000 UTC m=+157.211364546" watchObservedRunningTime="2026-09-18 18:03:05.527320504 +0000 UTC m=+157.213949956"
Sep 18 18:03:27.027459 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.027383    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:03:27.029715 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.029698    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.044253 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.044229    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:03:27.129332 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129312    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129351    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129390    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx6q4\" (UniqueName: \"kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129460    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.129580 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.129512    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230534    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230560    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230582    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230599    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx6q4\" (UniqueName: \"kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230625    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230809 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230756    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.230920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.230807    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.231426 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.231401    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.231528 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.231408    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.231592 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.231526    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.231754 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.231735    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.232984 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.232955    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.233232 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.233211    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.239220 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.239202    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx6q4\" (UniqueName: \"kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4\") pod \"console-698fff4f6d-2647l\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") " pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.338787 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.338735    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:27.610707 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:27.610682    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:03:27.612963 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:03:27.612915    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252b1d06_c773_4489_aebb_2a73421772a9.slice/crio-218f8663f0b80b1adfb9e47c55a1ad308e73239d742af28f096a51dcb2aaac68 WatchSource:0}: Error finding container 218f8663f0b80b1adfb9e47c55a1ad308e73239d742af28f096a51dcb2aaac68: Status 404 returned error can't find the container with id 218f8663f0b80b1adfb9e47c55a1ad308e73239d742af28f096a51dcb2aaac68
Sep 18 18:03:28.588412 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:28.588378    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698fff4f6d-2647l" event={"ID":"252b1d06-c773-4489-aebb-2a73421772a9","Type":"ContainerStarted","Data":"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"}
Sep 18 18:03:28.588848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:28.588430    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698fff4f6d-2647l" event={"ID":"252b1d06-c773-4489-aebb-2a73421772a9","Type":"ContainerStarted","Data":"218f8663f0b80b1adfb9e47c55a1ad308e73239d742af28f096a51dcb2aaac68"}
Sep 18 18:03:28.621453 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:28.621405    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-698fff4f6d-2647l" podStartSLOduration=1.62139009 podStartE2EDuration="1.62139009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:03:28.620174849 +0000 UTC m=+180.306804302" watchObservedRunningTime="2026-09-18 18:03:28.62139009 +0000 UTC m=+180.308019540"
Sep 18 18:03:37.339396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:37.339346    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:37.339396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:37.339414    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:37.344221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:37.344196    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:37.629490 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:37.629414    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:03:37.678363 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:03:37.678333    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:04:02.698976 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.698913    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f9bdbfdc-vgsf5" podUID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" containerName="console" containerID="cri-o://b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff" gracePeriod=15
Sep 18 18:04:02.937145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.937117    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:04:02.981784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981728    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.981784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981763    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.981784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981783    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.982020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981804    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.982020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981877    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqjmh\" (UniqueName: \"kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.982020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981904    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.982020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.981930    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert\") pod \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\" (UID: \"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8\") "
Sep 18 18:04:02.982321 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.982293    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config" (OuterVolumeSpecName: "console-config") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:02.982402 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.982300    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca" (OuterVolumeSpecName: "service-ca") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:02.982572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.982539    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:02.982572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.982561    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:04:02.983883 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.983854    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:04:02.984013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.983893    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:04:02.984121 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:02.984105    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh" (OuterVolumeSpecName: "kube-api-access-hqjmh") pod "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" (UID: "bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8"). InnerVolumeSpecName "kube-api-access-hqjmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:04:03.083407 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083382    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083407 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083404    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-trusted-ca-bundle\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083419    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqjmh\" (UniqueName: \"kubernetes.io/projected/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-kube-api-access-hqjmh\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083431    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-oauth-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083443    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-oauth-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083455    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-console-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.083616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.083467    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:04:03.720270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.720240    2550 generic.go:356] "Generic (PLEG): container finished" podID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" containerID="b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff" exitCode=0
Sep 18 18:04:03.720626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.720292    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f9bdbfdc-vgsf5"
Sep 18 18:04:03.720626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.720298    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f9bdbfdc-vgsf5" event={"ID":"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8","Type":"ContainerDied","Data":"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"}
Sep 18 18:04:03.720626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.720335    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f9bdbfdc-vgsf5" event={"ID":"bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8","Type":"ContainerDied","Data":"a1937df4266aafca47a6ae74f9508d350e0487a8a2cc12bb717445f0d3cc8370"}
Sep 18 18:04:03.720626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.720357    2550 scope.go:117] "RemoveContainer" containerID="b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"
Sep 18 18:04:03.729852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.729834    2550 scope.go:117] "RemoveContainer" containerID="b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"
Sep 18 18:04:03.730181 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:04:03.730153    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff\": container with ID starting with b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff not found: ID does not exist" containerID="b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"
Sep 18 18:04:03.730242 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.730184    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff"} err="failed to get container status \"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff\": rpc error: code = NotFound desc = could not find container \"b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff\": container with ID starting with b1758309b78e23f54c46204cb571622c875158526ba3946f9239a226caa73bff not found: ID does not exist"
Sep 18 18:04:03.741685 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.741639    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:04:03.745909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:03.745888    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f9bdbfdc-vgsf5"]
Sep 18 18:04:04.705926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:04.705881    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" path="/var/lib/kubelet/pods/bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8/volumes"
Sep 18 18:04:09.409769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.409733    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"]
Sep 18 18:04:09.410186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.410035    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" containerName="console"
Sep 18 18:04:09.410186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.410049    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" containerName="console"
Sep 18 18:04:09.410186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.410108    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bacfb6b8-a9e7-4ac3-9d6a-2482c51e9da8" containerName="console"
Sep 18 18:04:09.412665 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.412650    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.416353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.416330    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8ljxc\""
Sep 18 18:04:09.416463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.416336    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:04:09.416898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.416878    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 18 18:04:09.428536 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.428509    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"]
Sep 18 18:04:09.429922 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.429902    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v2tp\" (UniqueName: \"kubernetes.io/projected/39343657-fb63-41d4-863a-0fd338f3e4b8-kube-api-access-6v2tp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.430031 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.429957    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39343657-fb63-41d4-863a-0fd338f3e4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.530525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.530501    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v2tp\" (UniqueName: \"kubernetes.io/projected/39343657-fb63-41d4-863a-0fd338f3e4b8-kube-api-access-6v2tp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.530679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.530543    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39343657-fb63-41d4-863a-0fd338f3e4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.530882 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.530867    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/39343657-fb63-41d4-863a-0fd338f3e4b8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.541655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.541631    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v2tp\" (UniqueName: \"kubernetes.io/projected/39343657-fb63-41d4-863a-0fd338f3e4b8-kube-api-access-6v2tp\") pod \"cert-manager-operator-controller-manager-55c444bf86-knw6n\" (UID: \"39343657-fb63-41d4-863a-0fd338f3e4b8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:09.721829 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:09.721791    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"
Sep 18 18:04:10.003273 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:10.003199    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n"]
Sep 18 18:04:10.006955 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:04:10.006914    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39343657_fb63_41d4_863a_0fd338f3e4b8.slice/crio-5df690f658e3fddc2c14164859d0d94315c5f91043ab6052421ba85180aaa432 WatchSource:0}: Error finding container 5df690f658e3fddc2c14164859d0d94315c5f91043ab6052421ba85180aaa432: Status 404 returned error can't find the container with id 5df690f658e3fddc2c14164859d0d94315c5f91043ab6052421ba85180aaa432
Sep 18 18:04:10.749515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:10.749483    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n" event={"ID":"39343657-fb63-41d4-863a-0fd338f3e4b8","Type":"ContainerStarted","Data":"5df690f658e3fddc2c14164859d0d94315c5f91043ab6052421ba85180aaa432"}
Sep 18 18:04:14.767690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:14.767655    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n" event={"ID":"39343657-fb63-41d4-863a-0fd338f3e4b8","Type":"ContainerStarted","Data":"c5f80a7bd50629a922414a2a3afbe319e35365be736312cc8cc6d9efa6a3831b"}
Sep 18 18:04:14.791260 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:14.791213    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-knw6n" podStartSLOduration=2.008230504 podStartE2EDuration="5.791193958s" totalImagesPullingTime="3.782963454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:14.789902658 +0000 UTC m=+226.476532128" watchObservedRunningTime="2026-09-18 18:04:14.791193958 +0000 UTC m=+226.477823410"
Sep 18 18:04:17.194363 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.194326    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"]
Sep 18 18:04:17.196473 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.196449    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.200954 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.200920    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 18:04:17.201813 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.201794    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-jqr4z\""
Sep 18 18:04:17.201905 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.201796    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 18:04:17.207884 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.207862    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"]
Sep 18 18:04:17.290633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.290609    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.290738 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.290720    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tgmx\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-kube-api-access-4tgmx\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.391660 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.391635    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tgmx\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-kube-api-access-4tgmx\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.391755 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.391671    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.400976 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.400949    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.401166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.401146    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tgmx\" (UniqueName: \"kubernetes.io/projected/90a79579-14d1-43c9-8857-35fdc806a1a2-kube-api-access-4tgmx\") pod \"cert-manager-cainjector-559cdd8c87-ppx8w\" (UID: \"90a79579-14d1-43c9-8857-35fdc806a1a2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.528356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.528337    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"
Sep 18 18:04:17.804092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:17.804056    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w"]
Sep 18 18:04:17.806541 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:04:17.806514    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90a79579_14d1_43c9_8857_35fdc806a1a2.slice/crio-27e0b057cac320c35db5bef5486d81702057d505603024d54c8ace6190937000 WatchSource:0}: Error finding container 27e0b057cac320c35db5bef5486d81702057d505603024d54c8ace6190937000: Status 404 returned error can't find the container with id 27e0b057cac320c35db5bef5486d81702057d505603024d54c8ace6190937000
Sep 18 18:04:18.785343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:18.785304    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w" event={"ID":"90a79579-14d1-43c9-8857-35fdc806a1a2","Type":"ContainerStarted","Data":"27e0b057cac320c35db5bef5486d81702057d505603024d54c8ace6190937000"}
Sep 18 18:04:22.803149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:22.803105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w" event={"ID":"90a79579-14d1-43c9-8857-35fdc806a1a2","Type":"ContainerStarted","Data":"5b4a12d4e75b2e02782aad62b70658c2c13d5d9896d073e7b5d4d987632d7506"}
Sep 18 18:04:22.823983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:22.823907    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ppx8w" podStartSLOduration=1.544779352 podStartE2EDuration="5.823892633s" totalImagesPullingTime="4.279113281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:22.822161616 +0000 UTC m=+234.508791069" watchObservedRunningTime="2026-09-18 18:04:22.823892633 +0000 UTC m=+234.510522083"
Sep 18 18:04:25.545292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.545255    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwk4b"]
Sep 18 18:04:25.547622 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.547603    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.561465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.561445    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-njvls\""
Sep 18 18:04:25.592193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.592163    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwk4b"]
Sep 18 18:04:25.652668 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.652647    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jtx\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-kube-api-access-f2jtx\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.652803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.652738    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.753617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.753592    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jtx\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-kube-api-access-f2jtx\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.753732 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.753656    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.762315 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.762296    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jtx\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-kube-api-access-f2jtx\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.762693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.762673    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b29dfd2e-c10f-4f78-81c4-d7b747afdfb2-bound-sa-token\") pod \"cert-manager-55b4d954d5-wwk4b\" (UID: \"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2\") " pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:25.856817 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:25.856763    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-wwk4b"
Sep 18 18:04:26.128652 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:26.128627    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-wwk4b"]
Sep 18 18:04:26.130898 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:04:26.130869    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb29dfd2e_c10f_4f78_81c4_d7b747afdfb2.slice/crio-f3b89febb4af3ee21439f2c5a6a0a0d2bacbce2465a1d434bce6386e20f5f91a WatchSource:0}: Error finding container f3b89febb4af3ee21439f2c5a6a0a0d2bacbce2465a1d434bce6386e20f5f91a: Status 404 returned error can't find the container with id f3b89febb4af3ee21439f2c5a6a0a0d2bacbce2465a1d434bce6386e20f5f91a
Sep 18 18:04:26.819638 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:26.819605    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wwk4b" event={"ID":"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2","Type":"ContainerStarted","Data":"22e439f8144c73ebb61fe56e8ba6379e6bba42a15ce56f45345e2253afa642f2"}
Sep 18 18:04:26.820055 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:26.819646    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-wwk4b" event={"ID":"b29dfd2e-c10f-4f78-81c4-d7b747afdfb2","Type":"ContainerStarted","Data":"f3b89febb4af3ee21439f2c5a6a0a0d2bacbce2465a1d434bce6386e20f5f91a"}
Sep 18 18:04:26.837217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:26.837172    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-wwk4b" podStartSLOduration=1.8371572459999999 podStartE2EDuration="1.837157246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:26.835706682 +0000 UTC m=+238.522336134" watchObservedRunningTime="2026-09-18 18:04:26.837157246 +0000 UTC m=+238.523786703"
Sep 18 18:04:49.190374 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.190299    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"]
Sep 18 18:04:49.194809 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.194791    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.198509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.198489    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 18:04:49.198620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.198489    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 18:04:49.198797 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.198783    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 18:04:49.199150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.199136    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cj7q6\""
Sep 18 18:04:49.204385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.204343    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 18:04:49.222585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.222557    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"]
Sep 18 18:04:49.312734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.312708    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.312734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.312737    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.312894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.312760    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.312894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.312814    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.312894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.312842    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tswhh\" (UniqueName: \"kubernetes.io/projected/b0a47825-8f15-4d6b-87bf-abc34350a349-kube-api-access-tswhh\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.413956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.413919    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.414076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.413962    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.414076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.413983    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.414076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.414006    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.414076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.414021    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tswhh\" (UniqueName: \"kubernetes.io/projected/b0a47825-8f15-4d6b-87bf-abc34350a349-kube-api-access-tswhh\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.414423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.414403    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-manifests\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.416318 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.416298    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0a47825-8f15-4d6b-87bf-abc34350a349-tmp\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.416389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.416341    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.416470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.416453    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0a47825-8f15-4d6b-87bf-abc34350a349-webhook-cert\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.446328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.446283    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tswhh\" (UniqueName: \"kubernetes.io/projected/b0a47825-8f15-4d6b-87bf-abc34350a349-kube-api-access-tswhh\") pod \"opendatahub-operator-controller-manager-57cbb766c6-cxxrk\" (UID: \"b0a47825-8f15-4d6b-87bf-abc34350a349\") " pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.505246 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.505228    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:49.791470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.791441    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"]
Sep 18 18:04:49.795428 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:04:49.795399    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a47825_8f15_4d6b_87bf_abc34350a349.slice/crio-db69e08dae335244782c2cccf55bde27ff9cc131645bc66ef48c7f3bcac070ec WatchSource:0}: Error finding container db69e08dae335244782c2cccf55bde27ff9cc131645bc66ef48c7f3bcac070ec: Status 404 returned error can't find the container with id db69e08dae335244782c2cccf55bde27ff9cc131645bc66ef48c7f3bcac070ec
Sep 18 18:04:49.920420 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:49.920386    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk" event={"ID":"b0a47825-8f15-4d6b-87bf-abc34350a349","Type":"ContainerStarted","Data":"db69e08dae335244782c2cccf55bde27ff9cc131645bc66ef48c7f3bcac070ec"}
Sep 18 18:04:52.935706 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:52.935682    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk" event={"ID":"b0a47825-8f15-4d6b-87bf-abc34350a349","Type":"ContainerStarted","Data":"577d6b719ed63276ee482d4f9d59547d066cca4e5a716e065f9c7546435ca7c6"}
Sep 18 18:04:54.944864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:54.944833    2550 generic.go:356] "Generic (PLEG): container finished" podID="b0a47825-8f15-4d6b-87bf-abc34350a349" containerID="577d6b719ed63276ee482d4f9d59547d066cca4e5a716e065f9c7546435ca7c6" exitCode=0
Sep 18 18:04:54.945257 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:54.944914    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk" event={"ID":"b0a47825-8f15-4d6b-87bf-abc34350a349","Type":"ContainerDied","Data":"577d6b719ed63276ee482d4f9d59547d066cca4e5a716e065f9c7546435ca7c6"}
Sep 18 18:04:55.951324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:55.951288    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk" event={"ID":"b0a47825-8f15-4d6b-87bf-abc34350a349","Type":"ContainerStarted","Data":"b59f6bbaf890e9347f5f872ae20868882e8c25448ffedc631fdd27a786e8a236"}
Sep 18 18:04:55.951686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:55.951498    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:04:55.981177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:04:55.981134    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk" podStartSLOduration=2.3842441770000002 podStartE2EDuration="6.98112145s" totalImagesPullingTime="3.538405766s" totalInitContainerRuntime="1.058471507s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:04:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:04:55.979494549 +0000 UTC m=+267.666124000" watchObservedRunningTime="2026-09-18 18:04:55.98112145 +0000 UTC m=+267.667750902"
Sep 18 18:05:06.958088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:06.958056    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57cbb766c6-cxxrk"
Sep 18 18:05:09.544326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.544295    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"]
Sep 18 18:05:09.549893 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.549871    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.553909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.553889    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 18:05:09.554092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.554077    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5q28j\""
Sep 18 18:05:09.554673 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.554658    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 18:05:09.554716 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.554680    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 18:05:09.554752 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.554740    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 18:05:09.573861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.573840    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"]
Sep 18 18:05:09.665986 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.665957    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11277bc3-e440-41ee-bf9e-277f4a46849d-tmp\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.666078 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.666007    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trqt8\" (UniqueName: \"kubernetes.io/projected/11277bc3-e440-41ee-bf9e-277f4a46849d-kube-api-access-trqt8\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.666078 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.666038    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.766512 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.766492    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11277bc3-e440-41ee-bf9e-277f4a46849d-tmp\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.766637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.766527    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trqt8\" (UniqueName: \"kubernetes.io/projected/11277bc3-e440-41ee-bf9e-277f4a46849d-kube-api-access-trqt8\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.766637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.766555    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.766752 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:05:09.766655    2550 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 18 18:05:09.766752 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:05:09.766712    2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs podName:11277bc3-e440-41ee-bf9e-277f4a46849d nodeName:}" failed. No retries permitted until 2026-09-18 18:05:10.266697017 +0000 UTC m=+281.953326447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs") pod "kube-auth-proxy-9fbf455f6-ssznq" (UID: "11277bc3-e440-41ee-bf9e-277f4a46849d") : secret "kube-auth-proxy-tls" not found
Sep 18 18:05:09.768692 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.768671    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11277bc3-e440-41ee-bf9e-277f4a46849d-tmp\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:09.777757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:09.777737    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trqt8\" (UniqueName: \"kubernetes.io/projected/11277bc3-e440-41ee-bf9e-277f4a46849d-kube-api-access-trqt8\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:10.271037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:10.270997    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:10.273431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:10.273409    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11277bc3-e440-41ee-bf9e-277f4a46849d-tls-certs\") pod \"kube-auth-proxy-9fbf455f6-ssznq\" (UID: \"11277bc3-e440-41ee-bf9e-277f4a46849d\") " pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:10.459602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:10.459567    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"
Sep 18 18:05:10.741132 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:10.741110    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq"]
Sep 18 18:05:10.743200 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:05:10.743170    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11277bc3_e440_41ee_bf9e_277f4a46849d.slice/crio-db92af11723befe9d1b05052bff96e9d3a2e4af36721ee3a3d7f3f2166f30cf8 WatchSource:0}: Error finding container db92af11723befe9d1b05052bff96e9d3a2e4af36721ee3a3d7f3f2166f30cf8: Status 404 returned error can't find the container with id db92af11723befe9d1b05052bff96e9d3a2e4af36721ee3a3d7f3f2166f30cf8
Sep 18 18:05:11.013686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.013654    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq" event={"ID":"11277bc3-e440-41ee-bf9e-277f4a46849d","Type":"ContainerStarted","Data":"db92af11723befe9d1b05052bff96e9d3a2e4af36721ee3a3d7f3f2166f30cf8"}
Sep 18 18:05:11.862453 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.862412    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:11.864928 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.864900    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.871044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.871015    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 18:05:11.871173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.871060    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 18:05:11.871402 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.871381    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-plh74\""
Sep 18 18:05:11.871524 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.871413    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 18:05:11.871593 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.871541    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:05:11.896902 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.896840    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:11.986275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986229    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986370    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986492    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwh9n\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986525    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986566    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:11.986715 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:11.986629    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.087833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.087794    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwh9n\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.087833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.087839    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.087880    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.087925    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.087975    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.088090    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.088139    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.088906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.088859    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.090773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.090729    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.091320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.091236    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.091320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.091281    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.091476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.091346    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.097971 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.097932    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.098517 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.098477    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwh9n\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n\") pod \"istiod-openshift-gateway-7b7995d5cd-vvd79\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.176588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.176517    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:12.472480 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:12.472411    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:13.708751 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:05:13.708720    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59617ca3_9d65_4a5d_8987_bb9098d28f6d.slice/crio-be02bc60ef5106325bae7e33eab4d40f97750ff73d90b32c1f76d78fbde6d0f1 WatchSource:0}: Error finding container be02bc60ef5106325bae7e33eab4d40f97750ff73d90b32c1f76d78fbde6d0f1: Status 404 returned error can't find the container with id be02bc60ef5106325bae7e33eab4d40f97750ff73d90b32c1f76d78fbde6d0f1
Sep 18 18:05:14.027079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:14.027046    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq" event={"ID":"11277bc3-e440-41ee-bf9e-277f4a46849d","Type":"ContainerStarted","Data":"671a084061be3f6f4a03ce32b4e469e4724df5d2de1e0dc7ad3e1a85397cdf50"}
Sep 18 18:05:14.028334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:14.028308    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" event={"ID":"59617ca3-9d65-4a5d-8987-bb9098d28f6d","Type":"ContainerStarted","Data":"be02bc60ef5106325bae7e33eab4d40f97750ff73d90b32c1f76d78fbde6d0f1"}
Sep 18 18:05:16.227797 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:16.227761    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:05:16.228194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:16.227849    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:05:17.046251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.046217    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" event={"ID":"59617ca3-9d65-4a5d-8987-bb9098d28f6d","Type":"ContainerStarted","Data":"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"}
Sep 18 18:05:17.046448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.046383    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:17.048170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.048145    2550 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-vvd79 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:05:17.048271 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.048192    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:05:17.073445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.073400    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" podStartSLOduration=3.556691966 podStartE2EDuration="6.073384978s" totalImagesPullingTime="2.516693012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:17.070213497 +0000 UTC m=+288.756842949" watchObservedRunningTime="2026-09-18 18:05:17.073384978 +0000 UTC m=+288.760014429"
Sep 18 18:05:17.074109 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:17.074081    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9fbf455f6-ssznq" podStartSLOduration=5.078732266 podStartE2EDuration="8.074069209s" totalImagesPullingTime="2.995336943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:14.051807961 +0000 UTC m=+285.738437432" watchObservedRunningTime="2026-09-18 18:05:17.074069209 +0000 UTC m=+288.760698665"
Sep 18 18:05:18.051525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:18.051497    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:28.663261 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.663227    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"]
Sep 18 18:05:28.669907 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.669883    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.674200 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674181    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 18:05:28.674324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674276    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 18:05:28.674382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674319    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fq68h\""
Sep 18 18:05:28.674382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674327    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 18:05:28.674663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674646    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 18:05:28.674765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.674669    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 18:05:28.683179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.683156    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"]
Sep 18 18:05:28.816400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.816365    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.816557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.816430    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-metrics-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.816557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.816462    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdb506b-6101-4677-b0d3-516804a6bdbc-manager-config\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.816557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.816480    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-892tk\" (UniqueName: \"kubernetes.io/projected/4fdb506b-6101-4677-b0d3-516804a6bdbc-kube-api-access-892tk\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.917554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.917489    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdb506b-6101-4677-b0d3-516804a6bdbc-manager-config\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.917554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.917520    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-892tk\" (UniqueName: \"kubernetes.io/projected/4fdb506b-6101-4677-b0d3-516804a6bdbc-kube-api-access-892tk\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.917554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.917546    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.917785 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.917612    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-metrics-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.918125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.918106    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fdb506b-6101-4677-b0d3-516804a6bdbc-manager-config\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.919786 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.919768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-metrics-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.920016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.919996    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fdb506b-6101-4677-b0d3-516804a6bdbc-cert\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.936849 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.936821    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-892tk\" (UniqueName: \"kubernetes.io/projected/4fdb506b-6101-4677-b0d3-516804a6bdbc-kube-api-access-892tk\") pod \"lws-controller-manager-6d5f876c6c-zndc7\" (UID: \"4fdb506b-6101-4677-b0d3-516804a6bdbc\") " pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:28.979564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:28.979542    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:29.260784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:29.260756    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"]
Sep 18 18:05:29.264089 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:05:29.264054    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fdb506b_6101_4677_b0d3_516804a6bdbc.slice/crio-321ac70c6a8e74906918b49d45d6ac91c3a4a117cda706162252ce70afaad33f WatchSource:0}: Error finding container 321ac70c6a8e74906918b49d45d6ac91c3a4a117cda706162252ce70afaad33f: Status 404 returned error can't find the container with id 321ac70c6a8e74906918b49d45d6ac91c3a4a117cda706162252ce70afaad33f
Sep 18 18:05:30.101285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:30.101249    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7" event={"ID":"4fdb506b-6101-4677-b0d3-516804a6bdbc","Type":"ContainerStarted","Data":"321ac70c6a8e74906918b49d45d6ac91c3a4a117cda706162252ce70afaad33f"}
Sep 18 18:05:30.645320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:30.645290    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:05:30.647409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:30.647383    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:05:30.649227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:30.649083    2550 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 18:05:32.111795 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:32.111759    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7" event={"ID":"4fdb506b-6101-4677-b0d3-516804a6bdbc","Type":"ContainerStarted","Data":"f89639746768071a48482aafe4066ca773678e82f80b8a02e255e56b65367358"}
Sep 18 18:05:32.112148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:32.111893    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:05:32.173925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:32.173880    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7" podStartSLOduration=1.4266853400000001 podStartE2EDuration="4.173866004s" totalImagesPullingTime="2.747180664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:05:32.167489888 +0000 UTC m=+303.854119359" watchObservedRunningTime="2026-09-18 18:05:32.173866004 +0000 UTC m=+303.860495456"
Sep 18 18:05:35.646290 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.646253    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:35.646671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.646453    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerName="discovery" containerID="cri-o://665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55" gracePeriod=30
Sep 18 18:05:35.884295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.884273    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:35.975142 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975087    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975142 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975120    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwh9n\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975296 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975145    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975296 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975165    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975401 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975300    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975401 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975387    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975495 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975415    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs\") pod \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\" (UID: \"59617ca3-9d65-4a5d-8987-bb9098d28f6d\") "
Sep 18 18:05:35.975555 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975522    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:05:35.975732 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.975712    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-ca-configmap\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:35.977482 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977456    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:35.977597 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977480    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:35.977742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977722    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n" (OuterVolumeSpecName: "kube-api-access-pwh9n") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "kube-api-access-pwh9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:35.977823 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977752    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token" (OuterVolumeSpecName: "istio-token") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:05:35.977881 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977832    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs" (OuterVolumeSpecName: "local-certs") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:05:35.977881 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:35.977847    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts" (OuterVolumeSpecName: "cacerts") pod "59617ca3-9d65-4a5d-8987-bb9098d28f6d" (UID: "59617ca3-9d65-4a5d-8987-bb9098d28f6d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:05:36.076731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076708    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwh9n\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-kube-api-access-pwh9n\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.076731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076731    2550 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-kubeconfig\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.076852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076744    2550 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-csr-dns-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.076852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076755    2550 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/59617ca3-9d65-4a5d-8987-bb9098d28f6d-cacerts\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.076852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076766    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59617ca3-9d65-4a5d-8987-bb9098d28f6d-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.076852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.076778    2550 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/59617ca3-9d65-4a5d-8987-bb9098d28f6d-local-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:05:36.129784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.129759    2550 generic.go:356] "Generic (PLEG): container finished" podID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerID="665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55" exitCode=0
Sep 18 18:05:36.129864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.129805    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"
Sep 18 18:05:36.129864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.129815    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" event={"ID":"59617ca3-9d65-4a5d-8987-bb9098d28f6d","Type":"ContainerDied","Data":"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"}
Sep 18 18:05:36.129864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.129853    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79" event={"ID":"59617ca3-9d65-4a5d-8987-bb9098d28f6d","Type":"ContainerDied","Data":"be02bc60ef5106325bae7e33eab4d40f97750ff73d90b32c1f76d78fbde6d0f1"}
Sep 18 18:05:36.129989 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.129872    2550 scope.go:117] "RemoveContainer" containerID="665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"
Sep 18 18:05:36.138102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.138088    2550 scope.go:117] "RemoveContainer" containerID="665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"
Sep 18 18:05:36.138382 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:05:36.138361    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55\": container with ID starting with 665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55 not found: ID does not exist" containerID="665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"
Sep 18 18:05:36.138428 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.138389    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55"} err="failed to get container status \"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55\": rpc error: code = NotFound desc = could not find container \"665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55\": container with ID starting with 665dc811368b43f7a702f1e7cc06757b8f69ada7cec26be17d92b97b24f41d55 not found: ID does not exist"
Sep 18 18:05:36.152077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.152053    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:36.163502 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.163475    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-vvd79"]
Sep 18 18:05:36.706928 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:36.706895    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" path="/var/lib/kubelet/pods/59617ca3-9d65-4a5d-8987-bb9098d28f6d/volumes"
Sep 18 18:05:43.118658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:05:43.118594    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d5f876c6c-zndc7"
Sep 18 18:06:50.035295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.035256    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:06:50.699043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.699011    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"]
Sep 18 18:06:50.699309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.699297    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerName="discovery"
Sep 18 18:06:50.699353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.699312    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerName="discovery"
Sep 18 18:06:50.699387 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.699375    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="59617ca3-9d65-4a5d-8987-bb9098d28f6d" containerName="discovery"
Sep 18 18:06:50.702375 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.702359    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.704751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.704722    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 18 18:06:50.704893 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.704846    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 18:06:50.705212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.705184    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 18 18:06:50.705321 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.705219    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 18:06:50.705752 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.705733    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j2d5b\""
Sep 18 18:06:50.715536 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.715516    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"]
Sep 18 18:06:50.809328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.809306    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0920464-7501-438b-a967-6b70a9cf9106-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.809445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.809335    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f67kp\" (UniqueName: \"kubernetes.io/projected/b0920464-7501-438b-a967-6b70a9cf9106-kube-api-access-f67kp\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.809445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.809364    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0920464-7501-438b-a967-6b70a9cf9106-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.910011 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.909985    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0920464-7501-438b-a967-6b70a9cf9106-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.910110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.910020    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f67kp\" (UniqueName: \"kubernetes.io/projected/b0920464-7501-438b-a967-6b70a9cf9106-kube-api-access-f67kp\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.910110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.910062    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0920464-7501-438b-a967-6b70a9cf9106-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.910675 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.910651    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b0920464-7501-438b-a967-6b70a9cf9106-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.912262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.912241    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b0920464-7501-438b-a967-6b70a9cf9106-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:50.922369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:50.922343    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f67kp\" (UniqueName: \"kubernetes.io/projected/b0920464-7501-438b-a967-6b70a9cf9106-kube-api-access-f67kp\") pod \"kuadrant-console-plugin-fbdd4f445-xzqtv\" (UID: \"b0920464-7501-438b-a967-6b70a9cf9106\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:51.011016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:51.010999    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"
Sep 18 18:06:51.289461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:51.289426    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv"]
Sep 18 18:06:51.291851 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:06:51.291819    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0920464_7501_438b_a967_6b70a9cf9106.slice/crio-0cc9a94c24c5c86aebe8550b6b467a422b80f888e3b93852b509e9ad1e571a01 WatchSource:0}: Error finding container 0cc9a94c24c5c86aebe8550b6b467a422b80f888e3b93852b509e9ad1e571a01: Status 404 returned error can't find the container with id 0cc9a94c24c5c86aebe8550b6b467a422b80f888e3b93852b509e9ad1e571a01
Sep 18 18:06:51.293236 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:51.293217    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:06:51.462977 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:51.462924    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv" event={"ID":"b0920464-7501-438b-a967-6b70a9cf9106","Type":"ContainerStarted","Data":"0cc9a94c24c5c86aebe8550b6b467a422b80f888e3b93852b509e9ad1e571a01"}
Sep 18 18:06:57.498513 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:57.498478    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv" event={"ID":"b0920464-7501-438b-a967-6b70a9cf9106","Type":"ContainerStarted","Data":"b80db1bff6c9d0a3e19bc545ebd157b46dd58096f0dc13857ef949a9881a3320"}
Sep 18 18:06:57.515985 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:06:57.515925    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-xzqtv" podStartSLOduration=1.956700762 podStartE2EDuration="7.51591073s" totalImagesPullingTime="5.559209968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:06:57.514819649 +0000 UTC m=+389.201449095" watchObservedRunningTime="2026-09-18 18:06:57.51591073 +0000 UTC m=+389.202540225"
Sep 18 18:07:15.056133 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.056075    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-698fff4f6d-2647l" podUID="252b1d06-c773-4489-aebb-2a73421772a9" containerName="console" containerID="cri-o://2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453" gracePeriod=15
Sep 18 18:07:15.294483 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.294455    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:07:15.387225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387166    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387217    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387242    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387299    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx6q4\" (UniqueName: \"kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387323    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387353    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387401    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca\") pod \"252b1d06-c773-4489-aebb-2a73421772a9\" (UID: \"252b1d06-c773-4489-aebb-2a73421772a9\") "
Sep 18 18:07:15.387613 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387566    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config" (OuterVolumeSpecName: "console-config") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:07:15.387673 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387626    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:07:15.387843 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387812    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:07:15.387973 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.387931    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca" (OuterVolumeSpecName: "service-ca") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:07:15.389425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.389397    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:07:15.389535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.389434    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:07:15.389535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.389471    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4" (OuterVolumeSpecName: "kube-api-access-bx6q4") pod "252b1d06-c773-4489-aebb-2a73421772a9" (UID: "252b1d06-c773-4489-aebb-2a73421772a9"). InnerVolumeSpecName "kube-api-access-bx6q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:07:15.487875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487849    2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.487875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487870    2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.487875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487879    2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-console-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.488059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487887    2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/252b1d06-c773-4489-aebb-2a73421772a9-console-oauth-config\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.488059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487897    2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-oauth-serving-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.488059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487905    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx6q4\" (UniqueName: \"kubernetes.io/projected/252b1d06-c773-4489-aebb-2a73421772a9-kube-api-access-bx6q4\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.488059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.487914    2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252b1d06-c773-4489-aebb-2a73421772a9-trusted-ca-bundle\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:15.580720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.580692    2550 generic.go:356] "Generic (PLEG): container finished" podID="252b1d06-c773-4489-aebb-2a73421772a9" containerID="2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453" exitCode=0
Sep 18 18:07:15.580816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.580734    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698fff4f6d-2647l"
Sep 18 18:07:15.580816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.580769    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698fff4f6d-2647l" event={"ID":"252b1d06-c773-4489-aebb-2a73421772a9","Type":"ContainerDied","Data":"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"}
Sep 18 18:07:15.580890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.580818    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698fff4f6d-2647l" event={"ID":"252b1d06-c773-4489-aebb-2a73421772a9","Type":"ContainerDied","Data":"218f8663f0b80b1adfb9e47c55a1ad308e73239d742af28f096a51dcb2aaac68"}
Sep 18 18:07:15.580890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.580841    2550 scope.go:117] "RemoveContainer" containerID="2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"
Sep 18 18:07:15.589932 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.589915    2550 scope.go:117] "RemoveContainer" containerID="2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"
Sep 18 18:07:15.590236 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:07:15.590200    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453\": container with ID starting with 2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453 not found: ID does not exist" containerID="2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"
Sep 18 18:07:15.590310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.590245    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453"} err="failed to get container status \"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453\": rpc error: code = NotFound desc = could not find container \"2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453\": container with ID starting with 2aaaf781e0e4b705a298aefe63851fecc235696674980de979c14fd9788a5453 not found: ID does not exist"
Sep 18 18:07:15.603593 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.603571    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:07:15.607426 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:15.607407    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-698fff4f6d-2647l"]
Sep 18 18:07:16.706350 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:16.706299    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="252b1d06-c773-4489-aebb-2a73421772a9" path="/var/lib/kubelet/pods/252b1d06-c773-4489-aebb-2a73421772a9/volumes"
Sep 18 18:07:33.711552 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.711520    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:07:33.711961 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.711852    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="252b1d06-c773-4489-aebb-2a73421772a9" containerName="console"
Sep 18 18:07:33.711961 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.711865    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="252b1d06-c773-4489-aebb-2a73421772a9" containerName="console"
Sep 18 18:07:33.711961 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.711921    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="252b1d06-c773-4489-aebb-2a73421772a9" containerName="console"
Sep 18 18:07:33.735017 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.734991    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:07:33.735158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.735081    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.738121 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.738098    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 18:07:33.742222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.742205    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:07:33.815634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.815612    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6725978e-bd86-406b-aa97-fd2d5d66ba32-config-file\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.815735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.815641    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qq2m\" (UniqueName: \"kubernetes.io/projected/6725978e-bd86-406b-aa97-fd2d5d66ba32-kube-api-access-4qq2m\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.916349 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.916321    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6725978e-bd86-406b-aa97-fd2d5d66ba32-config-file\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.916450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.916354    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qq2m\" (UniqueName: \"kubernetes.io/projected/6725978e-bd86-406b-aa97-fd2d5d66ba32-kube-api-access-4qq2m\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.916909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.916889    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6725978e-bd86-406b-aa97-fd2d5d66ba32-config-file\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:33.925137 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:33.925112    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qq2m\" (UniqueName: \"kubernetes.io/projected/6725978e-bd86-406b-aa97-fd2d5d66ba32-kube-api-access-4qq2m\") pod \"limitador-limitador-84994f7ffc-nrhnf\" (UID: \"6725978e-bd86-406b-aa97-fd2d5d66ba32\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:34.045443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.045414    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:34.144233 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.144196    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:34.147676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.147658    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:34.150213 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.150197    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-px4dt\""
Sep 18 18:07:34.156227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.156206    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:34.219554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.219531    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwc7z\" (UniqueName: \"kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z\") pod \"authorino-7bcfd8f9cd-t9p94\" (UID: \"e1fdb1c5-7f04-4df1-b737-74e75dde186c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:34.316285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.316231    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:07:34.319257 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:07:34.319234    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6725978e_bd86_406b_aa97_fd2d5d66ba32.slice/crio-d3fc026317567e870133876b9ae4fca1dab55e2cc38751e171ae498a3c0ff77e WatchSource:0}: Error finding container d3fc026317567e870133876b9ae4fca1dab55e2cc38751e171ae498a3c0ff77e: Status 404 returned error can't find the container with id d3fc026317567e870133876b9ae4fca1dab55e2cc38751e171ae498a3c0ff77e
Sep 18 18:07:34.320373 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.320344    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwc7z\" (UniqueName: \"kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z\") pod \"authorino-7bcfd8f9cd-t9p94\" (UID: \"e1fdb1c5-7f04-4df1-b737-74e75dde186c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:34.328523 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.328505    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwc7z\" (UniqueName: \"kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z\") pod \"authorino-7bcfd8f9cd-t9p94\" (UID: \"e1fdb1c5-7f04-4df1-b737-74e75dde186c\") " pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:34.456742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.456723    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:34.667931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.667852    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf" event={"ID":"6725978e-bd86-406b-aa97-fd2d5d66ba32","Type":"ContainerStarted","Data":"d3fc026317567e870133876b9ae4fca1dab55e2cc38751e171ae498a3c0ff77e"}
Sep 18 18:07:34.725915 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:34.725890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:34.728909 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:07:34.728885    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1fdb1c5_7f04_4df1_b737_74e75dde186c.slice/crio-135f64e18b856dacfe07d012e2bc6ed9afb9064fcd9f1b85e4dd0712c785e533 WatchSource:0}: Error finding container 135f64e18b856dacfe07d012e2bc6ed9afb9064fcd9f1b85e4dd0712c785e533: Status 404 returned error can't find the container with id 135f64e18b856dacfe07d012e2bc6ed9afb9064fcd9f1b85e4dd0712c785e533
Sep 18 18:07:35.674181 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:35.674148    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" event={"ID":"e1fdb1c5-7f04-4df1-b737-74e75dde186c","Type":"ContainerStarted","Data":"135f64e18b856dacfe07d012e2bc6ed9afb9064fcd9f1b85e4dd0712c785e533"}
Sep 18 18:07:37.688241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:37.688209    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf" event={"ID":"6725978e-bd86-406b-aa97-fd2d5d66ba32","Type":"ContainerStarted","Data":"af5ba6455216bcadeffc87c4694d6078e9e1d152882edfe85ea164b9cba24919"}
Sep 18 18:07:37.688630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:37.688354    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:07:37.707089 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:37.707041    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf" podStartSLOduration=1.567468668 podStartE2EDuration="4.707020126s" totalImagesPullingTime="3.139551458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:37.70502533 +0000 UTC m=+429.391654801" watchObservedRunningTime="2026-09-18 18:07:37.707020126 +0000 UTC m=+429.393649580"
Sep 18 18:07:37.714753 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:37.714686    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:38.694008 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:38.693966    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" event={"ID":"e1fdb1c5-7f04-4df1-b737-74e75dde186c","Type":"ContainerStarted","Data":"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"}
Sep 18 18:07:38.694436 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:38.694016    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" podUID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" containerName="authorino" containerID="cri-o://835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861" gracePeriod=30
Sep 18 18:07:38.712324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:38.712273    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" podStartSLOduration=1.4840803870000001 podStartE2EDuration="4.712252392s" totalImagesPullingTime="3.228172005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:07:38.710528178 +0000 UTC m=+430.397157630" watchObservedRunningTime="2026-09-18 18:07:38.712252392 +0000 UTC m=+430.398881846"
Sep 18 18:07:38.939837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:38.939811    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:39.061670 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.061646    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwc7z\" (UniqueName: \"kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z\") pod \"e1fdb1c5-7f04-4df1-b737-74e75dde186c\" (UID: \"e1fdb1c5-7f04-4df1-b737-74e75dde186c\") "
Sep 18 18:07:39.063730 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.063710    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z" (OuterVolumeSpecName: "kube-api-access-zwc7z") pod "e1fdb1c5-7f04-4df1-b737-74e75dde186c" (UID: "e1fdb1c5-7f04-4df1-b737-74e75dde186c"). InnerVolumeSpecName "kube-api-access-zwc7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:07:39.162315 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.162291    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwc7z\" (UniqueName: \"kubernetes.io/projected/e1fdb1c5-7f04-4df1-b737-74e75dde186c-kube-api-access-zwc7z\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:07:39.702603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.702566    2550 generic.go:356] "Generic (PLEG): container finished" podID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" containerID="835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861" exitCode=0
Sep 18 18:07:39.702994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.702608    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94"
Sep 18 18:07:39.702994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.702629    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" event={"ID":"e1fdb1c5-7f04-4df1-b737-74e75dde186c","Type":"ContainerDied","Data":"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"}
Sep 18 18:07:39.702994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.702674    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-t9p94" event={"ID":"e1fdb1c5-7f04-4df1-b737-74e75dde186c","Type":"ContainerDied","Data":"135f64e18b856dacfe07d012e2bc6ed9afb9064fcd9f1b85e4dd0712c785e533"}
Sep 18 18:07:39.702994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.702701    2550 scope.go:117] "RemoveContainer" containerID="835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"
Sep 18 18:07:39.711626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.711609    2550 scope.go:117] "RemoveContainer" containerID="835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"
Sep 18 18:07:39.711893 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:07:39.711875    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861\": container with ID starting with 835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861 not found: ID does not exist" containerID="835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"
Sep 18 18:07:39.712018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.711900    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861"} err="failed to get container status \"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861\": rpc error: code = NotFound desc = could not find container \"835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861\": container with ID starting with 835acd9b4074a0a1c1fed9354c9d9d9452e18d3958b4ba9873c05c6db42ff861 not found: ID does not exist"
Sep 18 18:07:39.728107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.728084    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:39.735197 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:39.735176    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-t9p94"]
Sep 18 18:07:40.706889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:40.706850    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" path="/var/lib/kubelet/pods/e1fdb1c5-7f04-4df1-b737-74e75dde186c/volumes"
Sep 18 18:07:48.694990 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:07:48.694907    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"
Sep 18 18:08:10.018277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.018246    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-fdl7l"]
Sep 18 18:08:10.018660 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.018541    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" containerName="authorino"
Sep 18 18:08:10.018660 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.018551    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" containerName="authorino"
Sep 18 18:08:10.018660 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.018604    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1fdb1c5-7f04-4df1-b737-74e75dde186c" containerName="authorino"
Sep 18 18:08:10.021711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.021690    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.024292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.024273    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-px4dt\""
Sep 18 18:08:10.029367 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.029341    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdl7l"]
Sep 18 18:08:10.064313 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.064289    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdl7l"]
Sep 18 18:08:10.064459 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:08:10.064443    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-c6m5r], unattached volumes=[], failed to process volumes=[kube-api-access-c6m5r]: context canceled" pod="kuadrant-system/authorino-557f876f69-fdl7l" podUID="eada147a-8cc5-4358-96a0-cd0c2d4be0e5"
Sep 18 18:08:10.096017 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.095996    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-b2zx6"]
Sep 18 18:08:10.099088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.099073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.101968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.101927    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 18:08:10.110706 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.110682    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-b2zx6"]
Sep 18 18:08:10.175432 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.175408    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6m5r\" (UniqueName: \"kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r\") pod \"authorino-557f876f69-fdl7l\" (UID: \"eada147a-8cc5-4358-96a0-cd0c2d4be0e5\") " pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.259622 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.259599    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-b2zx6"]
Sep 18 18:08:10.259773 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:08:10.259756    2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-79wzr tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-b2zx6" podUID="96902e91-2a3e-4698-a7df-324ce1f43f61"
Sep 18 18:08:10.276461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.276416    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.276519 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.276458    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6m5r\" (UniqueName: \"kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r\") pod \"authorino-557f876f69-fdl7l\" (UID: \"eada147a-8cc5-4358-96a0-cd0c2d4be0e5\") " pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.276519 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.276482    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wzr\" (UniqueName: \"kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.286777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.286758    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6m5r\" (UniqueName: \"kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r\") pod \"authorino-557f876f69-fdl7l\" (UID: \"eada147a-8cc5-4358-96a0-cd0c2d4be0e5\") " pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.289871 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.289852    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:08:10.293052 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.293037    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.301056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.301036    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:08:10.377430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.377408    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79wzr\" (UniqueName: \"kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.377540 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.377448    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.377540 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.377477    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqzmp\" (UniqueName: \"kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.377540 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.377525    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.379767 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.379748    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.385467 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.385441    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wzr\" (UniqueName: \"kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr\") pod \"authorino-6d4dfc649f-b2zx6\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") " pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.478442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.478418    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqzmp\" (UniqueName: \"kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.478544 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.478472    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.480465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.480447    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.487367 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.487344    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqzmp\" (UniqueName: \"kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp\") pod \"authorino-b78555db-twhlm\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") " pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.602116 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.602066    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:08:10.837011 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.836974    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.837159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.836977    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.862634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.862587    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:10.878784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.878766    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:10.879560 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.879540    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:08:10.882768 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:08:10.882745    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6035e7a3_2b8f_4d34_827e_462262e6eef7.slice/crio-8ce03768c393fca220b3459b73c80c570c348076fb34b539ab1a7c761f837e9e WatchSource:0}: Error finding container 8ce03768c393fca220b3459b73c80c570c348076fb34b539ab1a7c761f837e9e: Status 404 returned error can't find the container with id 8ce03768c393fca220b3459b73c80c570c348076fb34b539ab1a7c761f837e9e
Sep 18 18:08:10.981346 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.981322    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79wzr\" (UniqueName: \"kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr\") pod \"96902e91-2a3e-4698-a7df-324ce1f43f61\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") "
Sep 18 18:08:10.981437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.981354    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert\") pod \"96902e91-2a3e-4698-a7df-324ce1f43f61\" (UID: \"96902e91-2a3e-4698-a7df-324ce1f43f61\") "
Sep 18 18:08:10.981437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.981376    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6m5r\" (UniqueName: \"kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r\") pod \"eada147a-8cc5-4358-96a0-cd0c2d4be0e5\" (UID: \"eada147a-8cc5-4358-96a0-cd0c2d4be0e5\") "
Sep 18 18:08:10.983428 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.983403    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "96902e91-2a3e-4698-a7df-324ce1f43f61" (UID: "96902e91-2a3e-4698-a7df-324ce1f43f61"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:08:10.983510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.983450    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r" (OuterVolumeSpecName: "kube-api-access-c6m5r") pod "eada147a-8cc5-4358-96a0-cd0c2d4be0e5" (UID: "eada147a-8cc5-4358-96a0-cd0c2d4be0e5"). InnerVolumeSpecName "kube-api-access-c6m5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:08:10.983510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:10.983477    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr" (OuterVolumeSpecName: "kube-api-access-79wzr") pod "96902e91-2a3e-4698-a7df-324ce1f43f61" (UID: "96902e91-2a3e-4698-a7df-324ce1f43f61"). InnerVolumeSpecName "kube-api-access-79wzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:08:11.082639 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.082617    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79wzr\" (UniqueName: \"kubernetes.io/projected/96902e91-2a3e-4698-a7df-324ce1f43f61-kube-api-access-79wzr\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:08:11.082639 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.082636    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/96902e91-2a3e-4698-a7df-324ce1f43f61-tls-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:08:11.082972 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.082648    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6m5r\" (UniqueName: \"kubernetes.io/projected/eada147a-8cc5-4358-96a0-cd0c2d4be0e5-kube-api-access-c6m5r\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:08:11.843407 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.843375    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b78555db-twhlm" event={"ID":"6035e7a3-2b8f-4d34-827e-462262e6eef7","Type":"ContainerStarted","Data":"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"}
Sep 18 18:08:11.843407 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.843400    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-b2zx6"
Sep 18 18:08:11.843617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.843421    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b78555db-twhlm" event={"ID":"6035e7a3-2b8f-4d34-827e-462262e6eef7","Type":"ContainerStarted","Data":"8ce03768c393fca220b3459b73c80c570c348076fb34b539ab1a7c761f837e9e"}
Sep 18 18:08:11.843617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.843388    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-fdl7l"
Sep 18 18:08:11.862963 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.862909    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b78555db-twhlm" podStartSLOduration=1.327798578 podStartE2EDuration="1.862896892s" totalImagesPullingTime="535.098314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:08:11.861426699 +0000 UTC m=+463.548056151" watchObservedRunningTime="2026-09-18 18:08:11.862896892 +0000 UTC m=+463.549526345"
Sep 18 18:08:11.890585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.890562    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdl7l"]
Sep 18 18:08:11.900537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.900516    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-fdl7l"]
Sep 18 18:08:11.932752 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.932725    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-b2zx6"]
Sep 18 18:08:11.937630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:11.937601    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-b2zx6"]
Sep 18 18:08:12.707116 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:12.707078    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96902e91-2a3e-4698-a7df-324ce1f43f61" path="/var/lib/kubelet/pods/96902e91-2a3e-4698-a7df-324ce1f43f61/volumes"
Sep 18 18:08:12.707561 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:12.707378    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eada147a-8cc5-4358-96a0-cd0c2d4be0e5" path="/var/lib/kubelet/pods/eada147a-8cc5-4358-96a0-cd0c2d4be0e5/volumes"
Sep 18 18:08:56.007002 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:08:56.006959    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:09:09.190173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:09:09.190139    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:09:41.385681 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:09:41.385640    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:09:56.617392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:09:56.617358    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:10:15.292763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:15.292721    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:10:30.677005 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:30.676897    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:10:30.677005 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:30.676901    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:10:38.600733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.600689    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"]
Sep 18 18:10:38.605129 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.605108    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.607652 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.607626    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 18 18:10:38.608275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.608260    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 18:10:38.608744 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.608728    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 18:10:38.608804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.608739    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v6bqs\""
Sep 18 18:10:38.615105 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.615085    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"]
Sep 18 18:10:38.693083 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693056    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.693212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693110    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.693212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693198    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.693338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693256    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.693338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693289    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.693338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.693323    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tlx9\" (UniqueName: \"kubernetes.io/projected/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-kube-api-access-2tlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.793829 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.793805    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.793927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.793843    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.793927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.793893    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.793927    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.793980    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tlx9\" (UniqueName: \"kubernetes.io/projected/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-kube-api-access-2tlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.794019    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.794238    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.794320    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.794366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.794351    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.796158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.796135    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.796484 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.796466    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.802182 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.802165    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tlx9\" (UniqueName: \"kubernetes.io/projected/3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5-kube-api-access-2tlx9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj\" (UID: \"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:38.915493 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:38.915442    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:39.185447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:39.185410    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"]
Sep 18 18:10:39.187832 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:10:39.187809    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d512c3f_37a2_4f22_8e90_6bbe0cdc3fb5.slice/crio-f66e69dec456732f32d123295bb4e1ae239e1a260f8c13642c174e5a11f7c954 WatchSource:0}: Error finding container f66e69dec456732f32d123295bb4e1ae239e1a260f8c13642c174e5a11f7c954: Status 404 returned error can't find the container with id f66e69dec456732f32d123295bb4e1ae239e1a260f8c13642c174e5a11f7c954
Sep 18 18:10:39.482406 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:39.482333    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj" event={"ID":"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5","Type":"ContainerStarted","Data":"f66e69dec456732f32d123295bb4e1ae239e1a260f8c13642c174e5a11f7c954"}
Sep 18 18:10:40.299670 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:40.299623    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:10:40.490104 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:40.490048    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj" event={"ID":"3d512c3f-37a2-4f22-8e90-6bbe0cdc3fb5","Type":"ContainerStarted","Data":"95e40a8f5c1c14d535ed9ec444a0e1c191f8bf1fb0fa29e2a8e258bf7d2b83d1"}
Sep 18 18:10:40.509170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:40.509119    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj" podStartSLOduration=1.352793175 podStartE2EDuration="2.509103674s" totalImagesPullingTime="1.156310499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:10:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:10:40.508119601 +0000 UTC m=+612.194749077" watchObservedRunningTime="2026-09-18 18:10:40.509103674 +0000 UTC m=+612.195733516"
Sep 18 18:10:45.181997 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:45.181895    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:10:48.916080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:48.916044    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:48.916572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:48.916138    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:48.928327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:48.928305    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:49.543839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:49.543815    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdzjrj"
Sep 18 18:10:50.280291 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.280260    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"]
Sep 18 18:10:50.283571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.283554    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.286092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.286069    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 18 18:10:50.296551 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.296529    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"]
Sep 18 18:10:50.375557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375536    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.375657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375562    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.375657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375583    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bc812d-56e7-404f-be16-5a1221eac87d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.375657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375600    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkwj8\" (UniqueName: \"kubernetes.io/projected/69bc812d-56e7-404f-be16-5a1221eac87d-kube-api-access-mkwj8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.375818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375720    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.375818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.375793    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476424    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476458    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476476    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476497    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bc812d-56e7-404f-be16-5a1221eac87d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476512    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkwj8\" (UniqueName: \"kubernetes.io/projected/69bc812d-56e7-404f-be16-5a1221eac87d-kube-api-access-mkwj8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476546    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476856    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.476932 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476909    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.477042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.476955    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.478815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.478797    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69bc812d-56e7-404f-be16-5a1221eac87d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.479013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.478997    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69bc812d-56e7-404f-be16-5a1221eac87d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.484056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.484038    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkwj8\" (UniqueName: \"kubernetes.io/projected/69bc812d-56e7-404f-be16-5a1221eac87d-kube-api-access-mkwj8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq\" (UID: \"69bc812d-56e7-404f-be16-5a1221eac87d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.592246 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.592198    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:10:50.864717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:50.864695    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"]
Sep 18 18:10:50.866579 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:10:50.866553    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69bc812d_56e7_404f_be16_5a1221eac87d.slice/crio-267003241e5754d4a7f7991a64c061ae1dd93a342cc12b6395915f948ad9d15e WatchSource:0}: Error finding container 267003241e5754d4a7f7991a64c061ae1dd93a342cc12b6395915f948ad9d15e: Status 404 returned error can't find the container with id 267003241e5754d4a7f7991a64c061ae1dd93a342cc12b6395915f948ad9d15e
Sep 18 18:10:51.542948 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:51.542901    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq" event={"ID":"69bc812d-56e7-404f-be16-5a1221eac87d","Type":"ContainerStarted","Data":"e0ebabfa203ea3d9a9503853e454e63a0b7e41fde9a3f919134023ca58034501"}
Sep 18 18:10:51.543315 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:51.542956    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq" event={"ID":"69bc812d-56e7-404f-be16-5a1221eac87d","Type":"ContainerStarted","Data":"267003241e5754d4a7f7991a64c061ae1dd93a342cc12b6395915f948ad9d15e"}
Sep 18 18:10:51.566651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:51.566607    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq" podStartSLOduration=1.392102347 podStartE2EDuration="1.566592545s" totalImagesPullingTime="174.490198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:10:51.56538477 +0000 UTC m=+623.252014259" watchObservedRunningTime="2026-09-18 18:10:51.566592545 +0000 UTC m=+623.253222016"
Sep 18 18:10:52.496088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:10:52.496057    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:11:00.593093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:11:00.593060    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:11:00.593093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:11:00.593103    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:11:00.605190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:11:00.605165    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:11:01.597468 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:11:01.597441    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fxjrq"
Sep 18 18:12:56.775292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.775256    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b6f99bdff-xvbfq"]
Sep 18 18:12:56.778489 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.778472    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:56.797614 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.797589    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6f99bdff-xvbfq"]
Sep 18 18:12:56.897612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.897580    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-tls-cert\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:56.897791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.897634    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbc2j\" (UniqueName: \"kubernetes.io/projected/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-kube-api-access-dbc2j\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:56.998771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.998739    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-tls-cert\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:56.998968 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:56.998793    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbc2j\" (UniqueName: \"kubernetes.io/projected/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-kube-api-access-dbc2j\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:57.001181 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:57.001160    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-tls-cert\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:57.007477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:57.007450    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbc2j\" (UniqueName: \"kubernetes.io/projected/d9cf2dbb-81fd-4da3-afc7-b40bf0208287-kube-api-access-dbc2j\") pod \"authorino-b6f99bdff-xvbfq\" (UID: \"d9cf2dbb-81fd-4da3-afc7-b40bf0208287\") " pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:57.086972 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:57.086881    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b6f99bdff-xvbfq"
Sep 18 18:12:57.359349 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:57.359199    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b6f99bdff-xvbfq"]
Sep 18 18:12:57.362027 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:12:57.361995    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9cf2dbb_81fd_4da3_afc7_b40bf0208287.slice/crio-fa190d487bd01e8c2fe3b1c2183309749159c98cf28dd33a30ec9196b19916ec WatchSource:0}: Error finding container fa190d487bd01e8c2fe3b1c2183309749159c98cf28dd33a30ec9196b19916ec: Status 404 returned error can't find the container with id fa190d487bd01e8c2fe3b1c2183309749159c98cf28dd33a30ec9196b19916ec
Sep 18 18:12:57.363440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:57.363419    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:12:58.138518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:58.138484    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6f99bdff-xvbfq" event={"ID":"d9cf2dbb-81fd-4da3-afc7-b40bf0208287","Type":"ContainerStarted","Data":"fa190d487bd01e8c2fe3b1c2183309749159c98cf28dd33a30ec9196b19916ec"}
Sep 18 18:12:59.145539 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.145492    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b6f99bdff-xvbfq" event={"ID":"d9cf2dbb-81fd-4da3-afc7-b40bf0208287","Type":"ContainerStarted","Data":"c5e07227f93aded34d1a3661039bcf25552d3fbf308b21c6fabcf22d6842937d"}
Sep 18 18:12:59.168410 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.168350    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b6f99bdff-xvbfq" podStartSLOduration=2.165397323 podStartE2EDuration="3.168330062s" totalImagesPullingTime="1.002932739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:12:59.166364246 +0000 UTC m=+750.852993699" watchObservedRunningTime="2026-09-18 18:12:59.168330062 +0000 UTC m=+750.854959515"
Sep 18 18:12:59.213083 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.213049    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:12:59.213274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.213250    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b78555db-twhlm" podUID="6035e7a3-2b8f-4d34-827e-462262e6eef7" containerName="authorino" containerID="cri-o://b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4" gracePeriod=30
Sep 18 18:12:59.477396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.477372    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:12:59.521824 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.521795    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert\") pod \"6035e7a3-2b8f-4d34-827e-462262e6eef7\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") "
Sep 18 18:12:59.522015 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.521838    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqzmp\" (UniqueName: \"kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp\") pod \"6035e7a3-2b8f-4d34-827e-462262e6eef7\" (UID: \"6035e7a3-2b8f-4d34-827e-462262e6eef7\") "
Sep 18 18:12:59.523885 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.523851    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp" (OuterVolumeSpecName: "kube-api-access-fqzmp") pod "6035e7a3-2b8f-4d34-827e-462262e6eef7" (UID: "6035e7a3-2b8f-4d34-827e-462262e6eef7"). InnerVolumeSpecName "kube-api-access-fqzmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:12:59.531455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.531423    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6035e7a3-2b8f-4d34-827e-462262e6eef7" (UID: "6035e7a3-2b8f-4d34-827e-462262e6eef7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:12:59.623204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.623167    2550 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6035e7a3-2b8f-4d34-827e-462262e6eef7-tls-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:12:59.623204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:12:59.623199    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqzmp\" (UniqueName: \"kubernetes.io/projected/6035e7a3-2b8f-4d34-827e-462262e6eef7-kube-api-access-fqzmp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:13:00.152354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.152315    2550 generic.go:356] "Generic (PLEG): container finished" podID="6035e7a3-2b8f-4d34-827e-462262e6eef7" containerID="b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4" exitCode=0
Sep 18 18:13:00.152771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.152363    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b78555db-twhlm"
Sep 18 18:13:00.152771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.152404    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b78555db-twhlm" event={"ID":"6035e7a3-2b8f-4d34-827e-462262e6eef7","Type":"ContainerDied","Data":"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"}
Sep 18 18:13:00.152771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.152447    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b78555db-twhlm" event={"ID":"6035e7a3-2b8f-4d34-827e-462262e6eef7","Type":"ContainerDied","Data":"8ce03768c393fca220b3459b73c80c570c348076fb34b539ab1a7c761f837e9e"}
Sep 18 18:13:00.152771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.152468    2550 scope.go:117] "RemoveContainer" containerID="b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"
Sep 18 18:13:00.162243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.162225    2550 scope.go:117] "RemoveContainer" containerID="b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"
Sep 18 18:13:00.162506 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:13:00.162489    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4\": container with ID starting with b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4 not found: ID does not exist" containerID="b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"
Sep 18 18:13:00.162552 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.162514    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4"} err="failed to get container status \"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4\": rpc error: code = NotFound desc = could not find container \"b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4\": container with ID starting with b72b3ed50bdb8b01fc2a8e7883cdfde7892f7695bae2844c7a37f17641d974e4 not found: ID does not exist"
Sep 18 18:13:00.181046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.181017    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:13:00.189994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.189919    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b78555db-twhlm"]
Sep 18 18:13:00.706409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:13:00.706369    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6035e7a3-2b8f-4d34-827e-462262e6eef7" path="/var/lib/kubelet/pods/6035e7a3-2b8f-4d34-827e-462262e6eef7/volumes"
Sep 18 18:14:02.148631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.148546    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:14:02.149193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.149069    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6035e7a3-2b8f-4d34-827e-462262e6eef7" containerName="authorino"
Sep 18 18:14:02.149193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.149089    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6035e7a3-2b8f-4d34-827e-462262e6eef7" containerName="authorino"
Sep 18 18:14:02.149316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.149204    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6035e7a3-2b8f-4d34-827e-462262e6eef7" containerName="authorino"
Sep 18 18:14:02.152210 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.152189    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.155131 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.155109    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:14:02.155243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.155148    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-de7535-openshift-default-dockercfg-g6dth\""
Sep 18 18:14:02.169355 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.169329    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:14:02.200718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200694    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200725    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200750    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200775    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200793    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shqr\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200819    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.200860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200843    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.201138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.201138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.200932    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301508    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301574    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301604    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301628    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301699    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301904 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301839    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301902    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.301981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301962    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.301995    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shqr\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.302054    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.302089    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302398 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.302368    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.302423    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.302510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.302488    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.304044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.304014    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.304200 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.304184    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.310709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.310686    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.310841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.310723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shqr\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr\") pod \"e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") " pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.462120 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.462040    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:02.742211 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:02.742135    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:14:02.746410 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:02.746380    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96f900b_e64c_464d_8a37_294bcb6fafbc.slice/crio-acd72c662fc0524a9df2f30ecdc2a2250f1908d4ee619ded6ceef0998dd9b85f WatchSource:0}: Error finding container acd72c662fc0524a9df2f30ecdc2a2250f1908d4ee619ded6ceef0998dd9b85f: Status 404 returned error can't find the container with id acd72c662fc0524a9df2f30ecdc2a2250f1908d4ee619ded6ceef0998dd9b85f
Sep 18 18:14:03.451287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.451247    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerStarted","Data":"acd72c662fc0524a9df2f30ecdc2a2250f1908d4ee619ded6ceef0998dd9b85f"}
Sep 18 18:14:03.796162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.796120    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8906-a1a065"]
Sep 18 18:14:03.799541 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.799521    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:03.802087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.802061    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxj55\""
Sep 18 18:14:03.807203 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.807178    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8906-a1a065"]
Sep 18 18:14:03.916128 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:03.916098    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp5tp\" (UniqueName: \"kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp\") pod \"test-curl-8906-a1a065\" (UID: \"cd86554a-b9fe-435b-a742-f6171b61ca47\") " pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:04.016919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:04.016872    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp5tp\" (UniqueName: \"kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp\") pod \"test-curl-8906-a1a065\" (UID: \"cd86554a-b9fe-435b-a742-f6171b61ca47\") " pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:04.026582 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:04.026548    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp5tp\" (UniqueName: \"kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp\") pod \"test-curl-8906-a1a065\" (UID: \"cd86554a-b9fe-435b-a742-f6171b61ca47\") " pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:04.113316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:04.113238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:04.419265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:04.419179    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8906-a1a065"]
Sep 18 18:14:04.424761 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:04.424723    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd86554a_b9fe_435b_a742_f6171b61ca47.slice/crio-a6dd2849b6284a0adc8673996532edf55ad6715522c97842932a1135fe95d9d6 WatchSource:0}: Error finding container a6dd2849b6284a0adc8673996532edf55ad6715522c97842932a1135fe95d9d6: Status 404 returned error can't find the container with id a6dd2849b6284a0adc8673996532edf55ad6715522c97842932a1135fe95d9d6
Sep 18 18:14:04.457345 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:04.457311    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8906-a1a065" event={"ID":"cd86554a-b9fe-435b-a742-f6171b61ca47","Type":"ContainerStarted","Data":"a6dd2849b6284a0adc8673996532edf55ad6715522c97842932a1135fe95d9d6"}
Sep 18 18:14:05.687079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:05.687046    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:05.687347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:05.687124    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:05.687347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:05.687155    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:06.470433 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:06.470400    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerStarted","Data":"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"}
Sep 18 18:14:06.496440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:06.496341    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podStartSLOduration=1.557963059 podStartE2EDuration="4.496299126s" totalImagesPullingTime="2.938336067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:06.495909407 +0000 UTC m=+818.182538888" watchObservedRunningTime="2026-09-18 18:14:06.496299126 +0000 UTC m=+818.182928579"
Sep 18 18:14:07.462546 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.462453    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:07.467469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.467441    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:07.476572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.476539    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8906-a1a065" event={"ID":"cd86554a-b9fe-435b-a742-f6171b61ca47","Type":"ContainerStarted","Data":"e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4"}
Sep 18 18:14:07.476911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.476892    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:07.477760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.477739    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:07.563123 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:07.563069    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8906-a1a065" podStartSLOduration=1.866244668 podStartE2EDuration="4.563052378s" totalImagesPullingTime="2.69680771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:07.562814435 +0000 UTC m=+819.249443887" watchObservedRunningTime="2026-09-18 18:14:07.563052378 +0000 UTC m=+819.249681826"
Sep 18 18:14:08.218760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.218674    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8906-a1a065"]
Sep 18 18:14:08.225504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.225476    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8906-a1a065"]
Sep 18 18:14:08.349068 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.349026    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8906-fed45c"]
Sep 18 18:14:08.349468 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.349448    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" containerName="test-curl-8906-a1a065"
Sep 18 18:14:08.349562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.349472    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" containerName="test-curl-8906-a1a065"
Sep 18 18:14:08.349562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.349559    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" containerName="test-curl-8906-a1a065"
Sep 18 18:14:08.352447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.352424    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:08.364463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.364436    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8906-fed45c"]
Sep 18 18:14:08.463469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.463424    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qpnw\" (UniqueName: \"kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw\") pod \"test-curl-8906-fed45c\" (UID: \"3e23f548-2bab-49f9-86ff-58cfe3c26662\") " pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:08.481287 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:08.481206    2550 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8906-a1a065" container="test-curl-8906-a1a065"
Sep 18 18:14:08.564289 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.564251    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qpnw\" (UniqueName: \"kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw\") pod \"test-curl-8906-fed45c\" (UID: \"3e23f548-2bab-49f9-86ff-58cfe3c26662\") " pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:08.578141 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.578100    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qpnw\" (UniqueName: \"kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw\") pod \"test-curl-8906-fed45c\" (UID: \"3e23f548-2bab-49f9-86ff-58cfe3c26662\") " pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:08.661950 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.661912    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:08.998431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:08.998398    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8906-fed45c"]
Sep 18 18:14:09.002016 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:09.001987    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e23f548_2bab_49f9_86ff_58cfe3c26662.slice/crio-479a7f7cc76e2ddc8b73fe53f05c5c5e166bec915263fceba2275f65f844a3a5 WatchSource:0}: Error finding container 479a7f7cc76e2ddc8b73fe53f05c5c5e166bec915263fceba2275f65f844a3a5: Status 404 returned error can't find the container with id 479a7f7cc76e2ddc8b73fe53f05c5c5e166bec915263fceba2275f65f844a3a5
Sep 18 18:14:09.488450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:09.488412    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8906-fed45c" event={"ID":"3e23f548-2bab-49f9-86ff-58cfe3c26662","Type":"ContainerStarted","Data":"3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b"}
Sep 18 18:14:09.488450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:09.488459    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8906-fed45c" event={"ID":"3e23f548-2bab-49f9-86ff-58cfe3c26662","Type":"ContainerStarted","Data":"479a7f7cc76e2ddc8b73fe53f05c5c5e166bec915263fceba2275f65f844a3a5"}
Sep 18 18:14:09.488966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:09.488510    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8906-a1a065" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" containerName="test-curl-8906-a1a065" containerID="cri-o://e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4" gracePeriod=2
Sep 18 18:14:09.503207 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:09.503148    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8906-fed45c" podStartSLOduration=1.503131728 podStartE2EDuration="1.503131728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:09.502614947 +0000 UTC m=+821.189244424" watchObservedRunningTime="2026-09-18 18:14:09.503131728 +0000 UTC m=+821.189761183"
Sep 18 18:14:10.136039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.135947    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8906-fed45c"]
Sep 18 18:14:10.144823 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.144785    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8906-fed45c"]
Sep 18 18:14:10.493004 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:10.492900    2550 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8906-fed45c" container="test-curl-8906-fed45c"
Sep 18 18:14:10.910301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.910270    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:14:10.910714 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.910698    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" containerName="test-curl-8906-fed45c"
Sep 18 18:14:10.910767 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.910716    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" containerName="test-curl-8906-fed45c"
Sep 18 18:14:10.910802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.910787    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" containerName="test-curl-8906-fed45c"
Sep 18 18:14:10.913797 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.913782    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:10.916328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.916302    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-039dbc-dockercfg-xzbf6\""
Sep 18 18:14:10.916647 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.916633    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-039dbc\""
Sep 18 18:14:10.931379 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.931354    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:14:10.950572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.950538    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:14:10.953796 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.953780    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:10.964759 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.964736    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:14:10.985186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.985156    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj9px\" (UniqueName: \"kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:10.985344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:10.985204    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.086003 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.085958    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhksw\" (UniqueName: \"kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.086151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.086024    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj9px\" (UniqueName: \"kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.086151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.086078    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.086151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.086132    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.086825 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.086803    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.094569 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.094531    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj9px\" (UniqueName: \"kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px\") pod \"payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.187388 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.187302    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhksw\" (UniqueName: \"kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.187388 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.187360    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.188145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.188125    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.204128 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.204099    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhksw\" (UniqueName: \"kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw\") pod \"payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.223856 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.223821    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:11.264198 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.264159    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:11.499028 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.498969    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8906-fed45c" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" containerName="test-curl-8906-fed45c" containerID="cri-o://3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b" gracePeriod=2
Sep 18 18:14:11.500499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.500474    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:14:11.529990 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:11.529959    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ed5217_fc2c_4725_a1da_6b6a0b9136a0.slice/crio-d5402d16e50a2651106cc900a13ad5dae5bc21614e91053da29b53dc5ba82ee0 WatchSource:0}: Error finding container d5402d16e50a2651106cc900a13ad5dae5bc21614e91053da29b53dc5ba82ee0: Status 404 returned error can't find the container with id d5402d16e50a2651106cc900a13ad5dae5bc21614e91053da29b53dc5ba82ee0
Sep 18 18:14:11.547461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.547309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:14:11.552898 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:11.552868    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf446ffe6_f0d2_4708_b318_ceea6debf39c.slice/crio-d8c666a3cfd6465d6d90e3717130cd39a93f9f0973573d9d762e72d1b88f8046 WatchSource:0}: Error finding container d8c666a3cfd6465d6d90e3717130cd39a93f9f0973573d9d762e72d1b88f8046: Status 404 returned error can't find the container with id d8c666a3cfd6465d6d90e3717130cd39a93f9f0973573d9d762e72d1b88f8046
Sep 18 18:14:11.628516 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.628489    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:11.692090 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.692052    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp5tp\" (UniqueName: \"kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp\") pod \"cd86554a-b9fe-435b-a742-f6171b61ca47\" (UID: \"cd86554a-b9fe-435b-a742-f6171b61ca47\") "
Sep 18 18:14:11.694577 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.694545    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp" (OuterVolumeSpecName: "kube-api-access-pp5tp") pod "cd86554a-b9fe-435b-a742-f6171b61ca47" (UID: "cd86554a-b9fe-435b-a742-f6171b61ca47"). InnerVolumeSpecName "kube-api-access-pp5tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:14:11.793156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:11.793113    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp5tp\" (UniqueName: \"kubernetes.io/projected/cd86554a-b9fe-435b-a742-f6171b61ca47-kube-api-access-pp5tp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:14:12.093911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.093825    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:14:12.137445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.137410    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:14:12.141634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.141600    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.144244 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.143957    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-de7535-dockercfg-zw6nt\""
Sep 18 18:14:12.144244 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.143968    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-de7535\""
Sep 18 18:14:12.161703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.161675    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:14:12.298351 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.298299    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vdnp\" (UniqueName: \"kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.298538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.298366    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.400821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.399440    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vdnp\" (UniqueName: \"kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.400821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.399498    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.400821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.400410    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.426547 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.426074    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vdnp\" (UniqueName: \"kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp\") pod \"payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") " pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.466455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.466100    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:12.507250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.507151    2550 generic.go:356] "Generic (PLEG): container finished" podID="cd86554a-b9fe-435b-a742-f6171b61ca47" containerID="e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4" exitCode=137
Sep 18 18:14:12.507641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.507260    2550 scope.go:117] "RemoveContainer" containerID="e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4"
Sep 18 18:14:12.507641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.507424    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:12.510231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.510202    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" event={"ID":"f446ffe6-f0d2-4708-b318-ceea6debf39c","Type":"ContainerStarted","Data":"d8c666a3cfd6465d6d90e3717130cd39a93f9f0973573d9d762e72d1b88f8046"}
Sep 18 18:14:12.514280 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.514205    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" event={"ID":"87ed5217-fc2c-4725-a1da-6b6a0b9136a0","Type":"ContainerStarted","Data":"d5402d16e50a2651106cc900a13ad5dae5bc21614e91053da29b53dc5ba82ee0"}
Sep 18 18:14:12.523899 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:12.523844    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8906-a1a065\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" pod="opendatahub/test-curl-8906-a1a065"
Sep 18 18:14:12.707591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.707507    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd86554a-b9fe-435b-a742-f6171b61ca47" path="/var/lib/kubelet/pods/cd86554a-b9fe-435b-a742-f6171b61ca47/volumes"
Sep 18 18:14:12.776405 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.776289    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:14:12.856090 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:12.856035    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5668ea8e_9103_4e3b_b5cd_210043d60414.slice/crio-4480937d8b84188fd7bdb444c415e345d61c842c9427b59aaf4d2d8ea96507a0 WatchSource:0}: Error finding container 4480937d8b84188fd7bdb444c415e345d61c842c9427b59aaf4d2d8ea96507a0: Status 404 returned error can't find the container with id 4480937d8b84188fd7bdb444c415e345d61c842c9427b59aaf4d2d8ea96507a0
Sep 18 18:14:12.858603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.858578    2550 scope.go:117] "RemoveContainer" containerID="e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4"
Sep 18 18:14:12.858981 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:12.858931    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4\": container with ID starting with e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4 not found: ID does not exist" containerID="e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4"
Sep 18 18:14:12.859062 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:12.858997    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4"} err="failed to get container status \"e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4\": rpc error: code = NotFound desc = could not find container \"e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4\": container with ID starting with e6275035bf826f344d4a9e2d6e5564c6266741dda79f4a3f1e9c9040103f95c4 not found: ID does not exist"
Sep 18 18:14:13.523638 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.523596    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" event={"ID":"f446ffe6-f0d2-4708-b318-ceea6debf39c","Type":"ContainerStarted","Data":"44492a964ae22372fe879027489982ab4cdcbe284b0a6c40eb70e1aac98fa434"}
Sep 18 18:14:13.524017 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.523804    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:13.525247 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.525223    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" event={"ID":"87ed5217-fc2c-4725-a1da-6b6a0b9136a0","Type":"ContainerStarted","Data":"25636c6ab7aa7f3b716677eb1f0c5f5c831c07eda01c3a2ed6d1c77b0e92b4d0"}
Sep 18 18:14:13.525381 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.525300    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:13.526962 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.526927    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" event={"ID":"5668ea8e-9103-4e3b-b5cd-210043d60414","Type":"ContainerStarted","Data":"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"}
Sep 18 18:14:13.527055 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.526983    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" event={"ID":"5668ea8e-9103-4e3b-b5cd-210043d60414","Type":"ContainerStarted","Data":"4480937d8b84188fd7bdb444c415e345d61c842c9427b59aaf4d2d8ea96507a0"}
Sep 18 18:14:13.527106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.527055    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:13.569564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.569507    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" podStartSLOduration=2.203655642 podStartE2EDuration="3.569488371s" totalImagesPullingTime="1.365832729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:13.5647359 +0000 UTC m=+825.251365374" watchObservedRunningTime="2026-09-18 18:14:13.569488371 +0000 UTC m=+825.256117846"
Sep 18 18:14:13.603156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.603097    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" podStartSLOduration=1.287511756 podStartE2EDuration="1.603082405s" totalImagesPullingTime="315.570649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:13.602045232 +0000 UTC m=+825.288674702" watchObservedRunningTime="2026-09-18 18:14:13.603082405 +0000 UTC m=+825.289711856"
Sep 18 18:14:13.630058 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.629691    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" podStartSLOduration=2.238257244 podStartE2EDuration="3.629668673s" totalImagesPullingTime="1.391411429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:13.629130612 +0000 UTC m=+825.315760091" watchObservedRunningTime="2026-09-18 18:14:13.629668673 +0000 UTC m=+825.316298125"
Sep 18 18:14:13.703843 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.703819    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:13.813868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.813828    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qpnw\" (UniqueName: \"kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw\") pod \"3e23f548-2bab-49f9-86ff-58cfe3c26662\" (UID: \"3e23f548-2bab-49f9-86ff-58cfe3c26662\") "
Sep 18 18:14:13.815992 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.815964    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw" (OuterVolumeSpecName: "kube-api-access-8qpnw") pod "3e23f548-2bab-49f9-86ff-58cfe3c26662" (UID: "3e23f548-2bab-49f9-86ff-58cfe3c26662"). InnerVolumeSpecName "kube-api-access-8qpnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:14:13.915268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:13.915228    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qpnw\" (UniqueName: \"kubernetes.io/projected/3e23f548-2bab-49f9-86ff-58cfe3c26662-kube-api-access-8qpnw\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:14:14.532866 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.532824    2550 generic.go:356] "Generic (PLEG): container finished" podID="3e23f548-2bab-49f9-86ff-58cfe3c26662" containerID="3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b" exitCode=137
Sep 18 18:14:14.532866 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.532867    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:14.533410 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.532962    2550 scope.go:117] "RemoveContainer" containerID="3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b"
Sep 18 18:14:14.544384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.544330    2550 scope.go:117] "RemoveContainer" containerID="3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b"
Sep 18 18:14:14.545379 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:14.545307    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b\": container with ID starting with 3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b not found: ID does not exist" containerID="3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b"
Sep 18 18:14:14.545656 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.545404    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b"} err="failed to get container status \"3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b\": rpc error: code = NotFound desc = could not find container \"3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b\": container with ID starting with 3fb75b2866d1659a4aae2ef93795a001ee9b40ee24e12b5c1ae52562635b7b7b not found: ID does not exist"
Sep 18 18:14:14.546092 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:14:14.546028    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-8906-fed45c\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" pod="opendatahub/test-curl-8906-fed45c"
Sep 18 18:14:14.706543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:14.706508    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e23f548-2bab-49f9-86ff-58cfe3c26662" path="/var/lib/kubelet/pods/3e23f548-2bab-49f9-86ff-58cfe3c26662/volumes"
Sep 18 18:14:24.123352 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.123316    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:14:24.126612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.126593    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.129731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.129713    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bcf02f-openshift-default-dockercfg-fd4t4\""
Sep 18 18:14:24.151958 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.151229    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:14:24.310827 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.310777    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.310827 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.310828    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wjw4\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.310868    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.310910    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.310951    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.311063    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.311090    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.311113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.311173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.311131    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412550    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412596    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412622    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412739    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412789    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412814    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wjw4\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412903    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.412967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.412961    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.413253 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.413035    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.413420 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.413391    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.413498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.413418    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.413545 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.413513    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.413659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.413629    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.415262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.415237    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.415461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.415441    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.427176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.427154    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.427830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.427812    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wjw4\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4\") pod \"e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") " pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.436733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.436715    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:24.535708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.535676    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:14:24.535862 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.535774    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:14:24.535862 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.535826    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:14:24.753643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.753129    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:14:24.768652 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:24.768613    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9471350e_1cae_41bc_968e_d55dda73e6b3.slice/crio-81ba9b2d8d7dc6eebca850994c51b4c63dfd0c58106e2fad47fd9ec9eb9ec310 WatchSource:0}: Error finding container 81ba9b2d8d7dc6eebca850994c51b4c63dfd0c58106e2fad47fd9ec9eb9ec310: Status 404 returned error can't find the container with id 81ba9b2d8d7dc6eebca850994c51b4c63dfd0c58106e2fad47fd9ec9eb9ec310
Sep 18 18:14:24.773983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.773905    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:24.774103 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.774041    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:24.774103 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:24.774083    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:25.589876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:25.589836    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerStarted","Data":"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"}
Sep 18 18:14:25.590367 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:25.589896    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerStarted","Data":"81ba9b2d8d7dc6eebca850994c51b4c63dfd0c58106e2fad47fd9ec9eb9ec310"}
Sep 18 18:14:25.621705 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:25.621652    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podStartSLOduration=1.621635134 podStartE2EDuration="1.621635134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:25.619854541 +0000 UTC m=+837.306484020" watchObservedRunningTime="2026-09-18 18:14:25.621635134 +0000 UTC m=+837.308264648"
Sep 18 18:14:26.437590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:26.437553    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:26.447551 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:26.447512    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:26.595282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:26.595243    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:26.596247 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:26.596224    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:28.996915 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:28.996871    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:14:30.616227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.616136    2550 generic.go:356] "Generic (PLEG): container finished" podID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerID="aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea" exitCode=0
Sep 18 18:14:30.616227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.616179    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerDied","Data":"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"}
Sep 18 18:14:30.616671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.616502    2550 scope.go:117] "RemoveContainer" containerID="aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"
Sep 18 18:14:30.619186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.619151    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:30.619283 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.619226    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:30.619283 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:30.619259    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:31.622757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.622723    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerStarted","Data":"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"}
Sep 18 18:14:31.911956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.911855    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:14:31.915092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.915073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:31.917526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.917498    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-bcf02f\""
Sep 18 18:14:31.917645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.917530    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-bcf02f-dockercfg-87rs5\""
Sep 18 18:14:31.924648 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.924622    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:14:31.987888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.987843    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:31.988103 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:31.987919    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpb29\" (UniqueName: \"kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.089399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.089359    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.089592 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.089437    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpb29\" (UniqueName: \"kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.090153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.090131    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.097256 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.097229    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpb29\" (UniqueName: \"kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29\") pod \"payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.226957 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.226836    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:32.466509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.466386    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:32.467425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.467356    2550 patch_prober.go:28] interesting pod/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 18 18:14:32.467625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.467555    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 18 18:14:32.522439 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.522278    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:14:32.525344 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:14:32.525318    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28808bd_e43c_4f85_98eb_ea8f3818149c.slice/crio-c6d321098706fba17f449f152ec2a2ecb9e8142ddb7b7394af53e9ea1fffc2ea WatchSource:0}: Error finding container c6d321098706fba17f449f152ec2a2ecb9e8142ddb7b7394af53e9ea1fffc2ea: Status 404 returned error can't find the container with id c6d321098706fba17f449f152ec2a2ecb9e8142ddb7b7394af53e9ea1fffc2ea
Sep 18 18:14:32.629740 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.629691    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" event={"ID":"e28808bd-e43c-4f85-98eb-ea8f3818149c","Type":"ContainerStarted","Data":"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"}
Sep 18 18:14:32.630209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.629758    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" event={"ID":"e28808bd-e43c-4f85-98eb-ea8f3818149c","Type":"ContainerStarted","Data":"c6d321098706fba17f449f152ec2a2ecb9e8142ddb7b7394af53e9ea1fffc2ea"}
Sep 18 18:14:32.630209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:32.630028    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:33.463434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:33.463387    2550 patch_prober.go:28] interesting pod/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 18 18:14:33.463762 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:33.463460    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 18 18:14:34.462737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:34.462695    2550 patch_prober.go:28] interesting pod/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 18 18:14:34.463266 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:34.462771    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 18 18:14:35.463342 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:35.463298    2550 patch_prober.go:28] interesting pod/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 18 18:14:35.463995 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:35.463927    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 18 18:14:36.462954 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:36.462884    2550 patch_prober.go:28] interesting pod/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Sep 18 18:14:36.463169 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:36.462981    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Sep 18 18:14:37.467601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:37.467565    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:37.468106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:37.467975    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:37.469123 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:37.469101    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:14:37.490169 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:37.490113    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" podStartSLOduration=6.4900955 podStartE2EDuration="6.4900955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:14:32.649486395 +0000 UTC m=+844.336115860" watchObservedRunningTime="2026-09-18 18:14:37.4900955 +0000 UTC m=+849.176724997"
Sep 18 18:14:42.874732 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:42.874695    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:14:43.670190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:43.670141    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:14:48.721125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.721089    2550 generic.go:356] "Generic (PLEG): container finished" podID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerID="e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48" exitCode=0
Sep 18 18:14:48.721611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.721162    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerDied","Data":"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"}
Sep 18 18:14:48.721611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.721598    2550 scope.go:117] "RemoveContainer" containerID="e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"
Sep 18 18:14:48.722713 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.722688    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:48.722774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.722754    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:48.722811 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:48.722784    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:14:49.727724 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:49.727687    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerStarted","Data":"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"}
Sep 18 18:14:50.437481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:50.437438    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:50.437733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:50.437710    2550 patch_prober.go:28] interesting pod/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 18:14:50.437818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:50.437770    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 18:14:51.437817 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:51.437779    2550 patch_prober.go:28] interesting pod/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 18:14:51.438243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:51.437838    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 18:14:52.437998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:52.437918    2550 patch_prober.go:28] interesting pod/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 18:14:52.438481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:52.438021    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 18:14:53.437993 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:53.437961    2550 patch_prober.go:28] interesting pod/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 18:14:53.438172 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:53.438015    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 18:14:54.437637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:54.437596    2550 patch_prober.go:28] interesting pod/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 18:14:54.437820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:54.437656    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 18:14:55.441291 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:55.441257    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:55.441741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:55.441565    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:14:55.442268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:14:55.442251    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:15:00.203018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.202984    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"]
Sep 18 18:15:00.205703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.205685    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"
Sep 18 18:15:00.208911 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:15:00.208884    2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" podUID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"
Sep 18 18:15:00.209210 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:15:00.209189    2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-cleanup-dockercfg-4ghvf\" is forbidden: User \"system:node:ip-10-0-129-128.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-129-128.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\"" type="*v1.Secret"
Sep 18 18:15:00.227342 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.227314    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"]
Sep 18 18:15:00.229733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.229718    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"
Sep 18 18:15:00.235821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.235796    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"]
Sep 18 18:15:00.246313 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:00.246290    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"]
Sep 18 18:15:01.216518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.216477    2550 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 18 18:15:01.216962 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.216552    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"
Sep 18 18:15:01.240769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.240740    2550 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 18 18:15:01.240907 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.240799    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"
Sep 18 18:15:01.487725 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.487694    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"]
Sep 18 18:15:01.489967 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:01.489913    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5257bc62_9cb0_47b2_b0dd_4d8b8c259386.slice/crio-1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059 WatchSource:0}: Error finding container 1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059: Status 404 returned error can't find the container with id 1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059
Sep 18 18:15:01.521206 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.521174    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"]
Sep 18 18:15:01.523108 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:01.523081    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b1a61c_6640_4cfb_b6b3_6ede7e2a0af1.slice/crio-473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251 WatchSource:0}: Error finding container 473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251: Status 404 returned error can't find the container with id 473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251
Sep 18 18:15:01.575369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.575336    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:15:01.793663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.793624    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" event={"ID":"5257bc62-9cb0-47b2-b0dd-4d8b8c259386","Type":"ContainerStarted","Data":"1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059"}
Sep 18 18:15:01.794821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:01.794792    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" event={"ID":"b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1","Type":"ContainerStarted","Data":"473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251"}
Sep 18 18:15:03.312598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.312562    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:03.319636 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.319610    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.322108 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.322078    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-2e816045\""
Sep 18 18:15:03.322287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.322156    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-2e816045-dockercfg-89866\""
Sep 18 18:15:03.325751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.325723    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:03.329486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.329463    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:03.332597 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.332575    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.345549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.345505    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:03.495106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.495068    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.495315 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.495142    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb7b8\" (UniqueName: \"kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.495440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.495217    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.495969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.495843    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76767\" (UniqueName: \"kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.597696 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.597608    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.597696 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.597662    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76767\" (UniqueName: \"kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.598041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.597720    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.598041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.597785    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb7b8\" (UniqueName: \"kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.598906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.598880    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.599262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.599225    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.606589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.606561    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb7b8\" (UniqueName: \"kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8\") pod \"payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.606804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.606776    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76767\" (UniqueName: \"kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767\") pod \"payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") " pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.633725 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.633505    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:03.644617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.644594    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:03.877162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.877127    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:15:03.955353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.955315    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:03.958758 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:03.958678    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d4deff_08e5_469e_9dba_c7516b8f9031.slice/crio-6d29219dbdf559e5c2f0304e2ae4f5423e434010dd42171ceb5945cdf513c7c5 WatchSource:0}: Error finding container 6d29219dbdf559e5c2f0304e2ae4f5423e434010dd42171ceb5945cdf513c7c5: Status 404 returned error can't find the container with id 6d29219dbdf559e5c2f0304e2ae4f5423e434010dd42171ceb5945cdf513c7c5
Sep 18 18:15:03.960957 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:03.960914    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:03.963200 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:03.963173    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3685e7a0_88a9_458c_96f9_48dc5e1d597a.slice/crio-fde687108ae53f1e5621f12f246acbd982f6edb58312076b6c76aba588128843 WatchSource:0}: Error finding container fde687108ae53f1e5621f12f246acbd982f6edb58312076b6c76aba588128843: Status 404 returned error can't find the container with id fde687108ae53f1e5621f12f246acbd982f6edb58312076b6c76aba588128843
Sep 18 18:15:04.815503 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.815462    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" event={"ID":"5257bc62-9cb0-47b2-b0dd-4d8b8c259386","Type":"ContainerStarted","Data":"9750ce8ec6695010dce1b886be8353d53c2d6d7d240fd8aff609478f104583ab"}
Sep 18 18:15:04.817085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.817059    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" event={"ID":"3685e7a0-88a9-458c-96f9-48dc5e1d597a","Type":"ContainerStarted","Data":"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"}
Sep 18 18:15:04.817247 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.817105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" event={"ID":"3685e7a0-88a9-458c-96f9-48dc5e1d597a","Type":"ContainerStarted","Data":"fde687108ae53f1e5621f12f246acbd982f6edb58312076b6c76aba588128843"}
Sep 18 18:15:04.817247 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.817126    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:04.818626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.818603    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" event={"ID":"76d4deff-08e5-469e-9dba-c7516b8f9031","Type":"ContainerStarted","Data":"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"}
Sep 18 18:15:04.818768 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.818645    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:04.818768 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.818667    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" event={"ID":"76d4deff-08e5-469e-9dba-c7516b8f9031","Type":"ContainerStarted","Data":"6d29219dbdf559e5c2f0304e2ae4f5423e434010dd42171ceb5945cdf513c7c5"}
Sep 18 18:15:04.820176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.820145    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" event={"ID":"b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1","Type":"ContainerStarted","Data":"b78873c6ef292cda1864f18a6f5d9c9586816ad18761f19c59ad920e5417199a"}
Sep 18 18:15:04.835353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.835310    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" podStartSLOduration=2.4770518360000002 podStartE2EDuration="4.835296976s" totalImagesPullingTime="2.35824514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:04.834104397 +0000 UTC m=+876.520733870" watchObservedRunningTime="2026-09-18 18:15:04.835296976 +0000 UTC m=+876.521926427"
Sep 18 18:15:04.857284 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.857239    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" podStartSLOduration=2.6709696469999997 podStartE2EDuration="4.857225595s" totalImagesPullingTime="2.186255948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:04.856223869 +0000 UTC m=+876.542853344" watchObservedRunningTime="2026-09-18 18:15:04.857225595 +0000 UTC m=+876.543855041"
Sep 18 18:15:04.873242 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.873203    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" podStartSLOduration=1.873189704 podStartE2EDuration="1.873189704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:04.872260566 +0000 UTC m=+876.558890040" watchObservedRunningTime="2026-09-18 18:15:04.873189704 +0000 UTC m=+876.559819204"
Sep 18 18:15:04.888183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:04.888142    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" podStartSLOduration=1.888128917 podStartE2EDuration="1.888128917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:04.887572066 +0000 UTC m=+876.574201510" watchObservedRunningTime="2026-09-18 18:15:04.888128917 +0000 UTC m=+876.574758369"
Sep 18 18:15:05.826763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:05.826730    2550 generic.go:356] "Generic (PLEG): container finished" podID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" containerID="9750ce8ec6695010dce1b886be8353d53c2d6d7d240fd8aff609478f104583ab" exitCode=0
Sep 18 18:15:05.827249 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:05.826823    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" event={"ID":"5257bc62-9cb0-47b2-b0dd-4d8b8c259386","Type":"ContainerDied","Data":"9750ce8ec6695010dce1b886be8353d53c2d6d7d240fd8aff609478f104583ab"}
Sep 18 18:15:05.828300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:05.828275    2550 generic.go:356] "Generic (PLEG): container finished" podID="b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1" containerID="b78873c6ef292cda1864f18a6f5d9c9586816ad18761f19c59ad920e5417199a" exitCode=0
Sep 18 18:15:05.828442 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:05.828359    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" event={"ID":"b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1","Type":"ContainerDied","Data":"b78873c6ef292cda1864f18a6f5d9c9586816ad18761f19c59ad920e5417199a"}
Sep 18 18:15:06.991533 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:06.991505    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"
Sep 18 18:15:06.994178 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:06.994153    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"
Sep 18 18:15:07.846484 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.846455    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"
Sep 18 18:15:07.846709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.846450    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv" event={"ID":"b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1","Type":"ContainerDied","Data":"473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251"}
Sep 18 18:15:07.846709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.846591    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="473ea16c080a9e299e0248605c393053e2e68d4c17fec940977aa13008e3c251"
Sep 18 18:15:07.847913 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.847890    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh" event={"ID":"5257bc62-9cb0-47b2-b0dd-4d8b8c259386","Type":"ContainerDied","Data":"1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059"}
Sep 18 18:15:07.848040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.847929    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1339ae9a86969b4b02394cbf149e0d8124240a2ba8614f713f68d82fc95a4059"
Sep 18 18:15:07.848040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:07.847932    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"
Sep 18 18:15:08.867416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.867381    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:15:08.867987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.867971    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" containerName="cleanup"
Sep 18 18:15:08.868033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.867991    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" containerName="cleanup"
Sep 18 18:15:08.868033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.868019    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1" containerName="cleanup"
Sep 18 18:15:08.868033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.868027    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1" containerName="cleanup"
Sep 18 18:15:08.868136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.868116    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1" containerName="cleanup"
Sep 18 18:15:08.868136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.868131    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" containerName="cleanup"
Sep 18 18:15:08.870727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.870704    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:08.873877 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.873849    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"default-dockercfg-82vrd\""
Sep 18 18:15:08.874039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.873874    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"openshift-service-ca.crt\""
Sep 18 18:15:08.874039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.873851    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"simulator-w0-37c153-kserve-self-signed-certs\""
Sep 18 18:15:08.874039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.873855    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"kube-root-ca.crt\""
Sep 18 18:15:08.879165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:08.879142    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:15:09.051343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051306    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.051571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051381    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md7zv\" (UniqueName: \"kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.051571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051496    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.051571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051540    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.051571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051570    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.051791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.051605    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152191    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md7zv\" (UniqueName: \"kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152297    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152315    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152332    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152400    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152736    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152760    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.152802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.152785    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.154788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.154766    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.154979 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.154961    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.160535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.160511    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md7zv\" (UniqueName: \"kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv\") pod \"simulator-w0-37c153-kserve-85544d9bc-vh97q\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") " pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.180542 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.180518    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:09.452286 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.452260    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:15:09.454461 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:09.454435    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda58b3dd_9c49_4e42_a945_b6e5cfa1feb6.slice/crio-4dcff2298631f011367dbf74bdc9af1f70f5ff0bc437951141f5186a2d29cf85 WatchSource:0}: Error finding container 4dcff2298631f011367dbf74bdc9af1f70f5ff0bc437951141f5186a2d29cf85: Status 404 returned error can't find the container with id 4dcff2298631f011367dbf74bdc9af1f70f5ff0bc437951141f5186a2d29cf85
Sep 18 18:15:09.862510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.862468    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" event={"ID":"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6","Type":"ContainerStarted","Data":"e32da859fe02e8acc23b129673c6685a7553b57b0e47dc659c40c8d5f299bdb2"}
Sep 18 18:15:09.862707 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.862523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" event={"ID":"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6","Type":"ContainerStarted","Data":"4dcff2298631f011367dbf74bdc9af1f70f5ff0bc437951141f5186a2d29cf85"}
Sep 18 18:15:09.880206 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:09.880158    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" podStartSLOduration=1.880140381 podStartE2EDuration="1.880140381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:09.87801284 +0000 UTC m=+881.564642312" watchObservedRunningTime="2026-09-18 18:15:09.880140381 +0000 UTC m=+881.566769832"
Sep 18 18:15:15.831204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:15.831116    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:15.831204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:15.831191    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:19.181097 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:19.181049    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:19.181528 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:19.181118    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:19.194054 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:19.194028    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:19.929369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:19.929339    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:15:24.099507 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:24.099467    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:15:30.713439 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:30.713412    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:15:30.713845 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:30.713744    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:15:31.582651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.582606    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:15:31.590400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.590373    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.593896 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.593869    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"default-dockercfg-q6jw7\""
Sep 18 18:15:31.594112 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.593896    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"kube-root-ca.crt\""
Sep 18 18:15:31.594282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.593907    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"simulator-w3-039dbc-kserve-self-signed-certs\""
Sep 18 18:15:31.594396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.594035    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"openshift-service-ca.crt\""
Sep 18 18:15:31.594456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.594390    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:15:31.663747 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663714    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdjcl\" (UniqueName: \"kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.663897 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663759    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.663897 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663803    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.663897 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663836    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.663897 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663880    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.664086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.663898    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.765515 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765479    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.766409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765527    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.766409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765595    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdjcl\" (UniqueName: \"kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.766409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765644    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.766409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765720    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.766409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.765750    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.767358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.767258    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.768294 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.768207    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.768626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.768568    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.774602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.774471    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.775309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.775242    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.777396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.777371    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdjcl\" (UniqueName: \"kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl\") pod \"simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") " pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:31.902227 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:31.902144    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:32.175319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:32.175248    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:15:32.177678 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:32.177651    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4794c02_448d_44ca_8d43_a308cb3be0ea.slice/crio-df61e6e1dc5770b4271f0dd00ac2ad39da2fe70c6c778684881a997037f7920c WatchSource:0}: Error finding container df61e6e1dc5770b4271f0dd00ac2ad39da2fe70c6c778684881a997037f7920c: Status 404 returned error can't find the container with id df61e6e1dc5770b4271f0dd00ac2ad39da2fe70c6c778684881a997037f7920c
Sep 18 18:15:32.983065 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:32.983025    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" event={"ID":"c4794c02-448d-44ca-8d43-a308cb3be0ea","Type":"ContainerStarted","Data":"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"}
Sep 18 18:15:32.983065 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:32.983072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" event={"ID":"c4794c02-448d-44ca-8d43-a308cb3be0ea","Type":"ContainerStarted","Data":"df61e6e1dc5770b4271f0dd00ac2ad39da2fe70c6c778684881a997037f7920c"}
Sep 18 18:15:32.999102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:32.999053    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" podStartSLOduration=1.999037683 podStartE2EDuration="1.999037683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:32.99804042 +0000 UTC m=+904.684669915" watchObservedRunningTime="2026-09-18 18:15:32.999037683 +0000 UTC m=+904.685667135"
Sep 18 18:15:33.911807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:33.911770    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:33.912201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:33.912047    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" podUID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" containerName="payload-processing" containerID="cri-o://c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644" gracePeriod=30
Sep 18 18:15:33.917418 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:33.917389    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:33.917752 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:33.917656    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" podUID="76d4deff-08e5-469e-9dba-c7516b8f9031" containerName="payload-pre-processing" containerID="cri-o://91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2" gracePeriod=30
Sep 18 18:15:34.194495 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.194470    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:34.197693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.197668    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:34.292468 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.292428    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76767\" (UniqueName: \"kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767\") pod \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") "
Sep 18 18:15:34.292734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.292522    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume\") pod \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\" (UID: \"3685e7a0-88a9-458c-96f9-48dc5e1d597a\") "
Sep 18 18:15:34.292734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.292632    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb7b8\" (UniqueName: \"kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8\") pod \"76d4deff-08e5-469e-9dba-c7516b8f9031\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") "
Sep 18 18:15:34.292734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.292672    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume\") pod \"76d4deff-08e5-469e-9dba-c7516b8f9031\" (UID: \"76d4deff-08e5-469e-9dba-c7516b8f9031\") "
Sep 18 18:15:34.292923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.292900    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3685e7a0-88a9-458c-96f9-48dc5e1d597a" (UID: "3685e7a0-88a9-458c-96f9-48dc5e1d597a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:15:34.293066 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.293048    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3685e7a0-88a9-458c-96f9-48dc5e1d597a-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:15:34.293151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.293125    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76d4deff-08e5-469e-9dba-c7516b8f9031" (UID: "76d4deff-08e5-469e-9dba-c7516b8f9031"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:15:34.294772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.294738    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767" (OuterVolumeSpecName: "kube-api-access-76767") pod "3685e7a0-88a9-458c-96f9-48dc5e1d597a" (UID: "3685e7a0-88a9-458c-96f9-48dc5e1d597a"). InnerVolumeSpecName "kube-api-access-76767". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:15:34.294772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.294749    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8" (OuterVolumeSpecName: "kube-api-access-cb7b8") pod "76d4deff-08e5-469e-9dba-c7516b8f9031" (UID: "76d4deff-08e5-469e-9dba-c7516b8f9031"). InnerVolumeSpecName "kube-api-access-cb7b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:15:34.394024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.393990    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76767\" (UniqueName: \"kubernetes.io/projected/3685e7a0-88a9-458c-96f9-48dc5e1d597a-kube-api-access-76767\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:15:34.394024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.394016    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb7b8\" (UniqueName: \"kubernetes.io/projected/76d4deff-08e5-469e-9dba-c7516b8f9031-kube-api-access-cb7b8\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:15:34.394024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.394027    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d4deff-08e5-469e-9dba-c7516b8f9031-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:15:34.995134 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.995042    2550 generic.go:356] "Generic (PLEG): container finished" podID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" containerID="c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644" exitCode=0
Sep 18 18:15:34.995134 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.995102    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"
Sep 18 18:15:34.995384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.995138    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" event={"ID":"3685e7a0-88a9-458c-96f9-48dc5e1d597a","Type":"ContainerDied","Data":"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"}
Sep 18 18:15:34.995384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.995233    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr" event={"ID":"3685e7a0-88a9-458c-96f9-48dc5e1d597a","Type":"ContainerDied","Data":"fde687108ae53f1e5621f12f246acbd982f6edb58312076b6c76aba588128843"}
Sep 18 18:15:34.995384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.995264    2550 scope.go:117] "RemoveContainer" containerID="c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"
Sep 18 18:15:34.996717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.996545    2550 generic.go:356] "Generic (PLEG): container finished" podID="76d4deff-08e5-469e-9dba-c7516b8f9031" containerID="91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2" exitCode=0
Sep 18 18:15:34.996717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.996603    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"
Sep 18 18:15:34.996717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.996609    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" event={"ID":"76d4deff-08e5-469e-9dba-c7516b8f9031","Type":"ContainerDied","Data":"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"}
Sep 18 18:15:34.996717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:34.996652    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7" event={"ID":"76d4deff-08e5-469e-9dba-c7516b8f9031","Type":"ContainerDied","Data":"6d29219dbdf559e5c2f0304e2ae4f5423e434010dd42171ceb5945cdf513c7c5"}
Sep 18 18:15:35.005820 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.005786    2550 scope.go:117] "RemoveContainer" containerID="c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"
Sep 18 18:15:35.006611 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:15:35.006577    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644\": container with ID starting with c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644 not found: ID does not exist" containerID="c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"
Sep 18 18:15:35.006804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.006634    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644"} err="failed to get container status \"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644\": rpc error: code = NotFound desc = could not find container \"c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644\": container with ID starting with c5b0a28d729376315f990e93294446a26e26ad1fbe940b2c69502cb2d086d644 not found: ID does not exist"
Sep 18 18:15:35.006804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.006688    2550 scope.go:117] "RemoveContainer" containerID="91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"
Sep 18 18:15:35.017073 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.017014    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:35.021726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.021696    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-2e816045-778864db74-lnxc7"]
Sep 18 18:15:35.023187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.023167    2550 scope.go:117] "RemoveContainer" containerID="91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"
Sep 18 18:15:35.023603 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:15:35.023573    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2\": container with ID starting with 91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2 not found: ID does not exist" containerID="91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"
Sep 18 18:15:35.023713 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.023615    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2"} err="failed to get container status \"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2\": rpc error: code = NotFound desc = could not find container \"91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2\": container with ID starting with 91ad38ca43628cb109c7c3ebfdbe978e0fd9650363c5fc3dd2a29798eef930b2 not found: ID does not exist"
Sep 18 18:15:35.037985 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.037929    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:35.044854 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:35.044825    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-2e816045-7b787dd846-twqwr"]
Sep 18 18:15:36.706425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:36.706387    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" path="/var/lib/kubelet/pods/3685e7a0-88a9-458c-96f9-48dc5e1d597a/volumes"
Sep 18 18:15:36.706815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:36.706759    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d4deff-08e5-469e-9dba-c7516b8f9031" path="/var/lib/kubelet/pods/76d4deff-08e5-469e-9dba-c7516b8f9031/volumes"
Sep 18 18:15:37.983864 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:37.983825    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:15:39.241215 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241171    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:15:39.241648 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241632    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d4deff-08e5-469e-9dba-c7516b8f9031" containerName="payload-pre-processing"
Sep 18 18:15:39.241694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241649    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d4deff-08e5-469e-9dba-c7516b8f9031" containerName="payload-pre-processing"
Sep 18 18:15:39.241694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241664    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" containerName="payload-processing"
Sep 18 18:15:39.241694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241670    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" containerName="payload-processing"
Sep 18 18:15:39.241787 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241743    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="3685e7a0-88a9-458c-96f9-48dc5e1d597a" containerName="payload-processing"
Sep 18 18:15:39.241787 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.241756    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d4deff-08e5-469e-9dba-c7516b8f9031" containerName="payload-pre-processing"
Sep 18 18:15:39.246351 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.246334    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.248647 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.248623    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-89dea9e1-dockercfg-ckfng\""
Sep 18 18:15:39.248894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.248873    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-89dea9e1\""
Sep 18 18:15:39.255892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.255868    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:15:39.342713 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.342667    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.342902 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.342788    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grrqq\" (UniqueName: \"kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.443576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.443544    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.443765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.443615    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grrqq\" (UniqueName: \"kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.444333 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.444309    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.451463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.451442    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grrqq\" (UniqueName: \"kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq\") pod \"payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") " pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.557678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.557646    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:39.831524 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:39.831442    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:15:39.834683 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:15:39.834656    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd755d48_1a0d_41b3_abe7_178686db84fe.slice/crio-44352e0c3debf8b10205f2dff9096cf356d23db9211b42686f6ea7fa74e60eb6 WatchSource:0}: Error finding container 44352e0c3debf8b10205f2dff9096cf356d23db9211b42686f6ea7fa74e60eb6: Status 404 returned error can't find the container with id 44352e0c3debf8b10205f2dff9096cf356d23db9211b42686f6ea7fa74e60eb6
Sep 18 18:15:40.027736 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:40.027698    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" event={"ID":"cd755d48-1a0d-41b3-abe7-178686db84fe","Type":"ContainerStarted","Data":"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"}
Sep 18 18:15:40.027912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:40.027757    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" event={"ID":"cd755d48-1a0d-41b3-abe7-178686db84fe","Type":"ContainerStarted","Data":"44352e0c3debf8b10205f2dff9096cf356d23db9211b42686f6ea7fa74e60eb6"}
Sep 18 18:15:40.027912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:40.027782    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:40.044741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:40.044683    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" podStartSLOduration=1.044666469 podStartE2EDuration="1.044666469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:15:40.042574316 +0000 UTC m=+911.729203805" watchObservedRunningTime="2026-09-18 18:15:40.044666469 +0000 UTC m=+911.731295921"
Sep 18 18:15:41.902437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:41.902379    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:41.902889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:41.902462    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:41.915149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:41.915125    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:42.050061 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:42.050027    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:15:51.036077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:51.036046    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:15:57.080421 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:15:57.080382    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:16:03.077646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.077610    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"]
Sep 18 18:16:03.081233 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.081212    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:03.083642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.083619    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:16:03.083789 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.083736    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:16:03.089438 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.089414    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"]
Sep 18 18:16:03.169430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.169394    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv\" (UID: \"e48b28e9-0687-4477-a142-cf8ab72744a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:03.270889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.270848    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv\" (UID: \"e48b28e9-0687-4477-a142-cf8ab72744a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:03.271541 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.271515    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv\" (UID: \"e48b28e9-0687-4477-a142-cf8ab72744a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:03.393812 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.393720    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:03.666172 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:03.666107    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"]
Sep 18 18:16:03.668679 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:16:03.668652    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode48b28e9_0687_4477_a142_cf8ab72744a4.slice/crio-1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c WatchSource:0}: Error finding container 1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c: Status 404 returned error can't find the container with id 1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c
Sep 18 18:16:04.161368 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:04.161324    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv" event={"ID":"e48b28e9-0687-4477-a142-cf8ab72744a4","Type":"ContainerStarted","Data":"35763ef421c524404a2c54ee0d24bb7d383c17873fcd7dd99284d0b397b14d36"}
Sep 18 18:16:04.161764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:04.161390    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv" event={"ID":"e48b28e9-0687-4477-a142-cf8ab72744a4","Type":"ContainerStarted","Data":"1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c"}
Sep 18 18:16:04.196245 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:04.196186    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv" podStartSLOduration=1.196169854 podStartE2EDuration="1.196169854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:16:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:16:04.194756882 +0000 UTC m=+935.881386334" watchObservedRunningTime="2026-09-18 18:16:04.196169854 +0000 UTC m=+935.882799306"
Sep 18 18:16:05.167898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:05.167859    2550 generic.go:356] "Generic (PLEG): container finished" podID="e48b28e9-0687-4477-a142-cf8ab72744a4" containerID="35763ef421c524404a2c54ee0d24bb7d383c17873fcd7dd99284d0b397b14d36" exitCode=0
Sep 18 18:16:05.168319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:05.167973    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv" event={"ID":"e48b28e9-0687-4477-a142-cf8ab72744a4","Type":"ContainerDied","Data":"35763ef421c524404a2c54ee0d24bb7d383c17873fcd7dd99284d0b397b14d36"}
Sep 18 18:16:06.312004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:06.311976    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:06.403221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:06.403188    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca\") pod \"e48b28e9-0687-4477-a142-cf8ab72744a4\" (UID: \"e48b28e9-0687-4477-a142-cf8ab72744a4\") "
Sep 18 18:16:06.403597 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:06.403575    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e48b28e9-0687-4477-a142-cf8ab72744a4" (UID: "e48b28e9-0687-4477-a142-cf8ab72744a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:16:06.504280 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:06.504184    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e48b28e9-0687-4477-a142-cf8ab72744a4-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:07.179201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:07.179169    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"
Sep 18 18:16:07.179384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:07.179207    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv" event={"ID":"e48b28e9-0687-4477-a142-cf8ab72744a4","Type":"ContainerDied","Data":"1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c"}
Sep 18 18:16:07.179384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:07.179259    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d5e2613251ea39b8d36ad80aef57652431b426ca65f9f9dcdc66e97c56f4e0c"
Sep 18 18:16:09.451230 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.451189    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:16:09.451626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.451458    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" podUID="cd755d48-1a0d-41b3-abe7-178686db84fe" containerName="payload-processing" containerID="cri-o://d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994" gracePeriod=30
Sep 18 18:16:09.638244 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.638183    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"]
Sep 18 18:16:09.640062 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.640035    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-89dea9e1-52jnv"]
Sep 18 18:16:09.696922 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.696897    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:16:09.833914 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.833826    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume\") pod \"cd755d48-1a0d-41b3-abe7-178686db84fe\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") "
Sep 18 18:16:09.833914 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.833880    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grrqq\" (UniqueName: \"kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq\") pod \"cd755d48-1a0d-41b3-abe7-178686db84fe\" (UID: \"cd755d48-1a0d-41b3-abe7-178686db84fe\") "
Sep 18 18:16:09.834320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.834294    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd755d48-1a0d-41b3-abe7-178686db84fe" (UID: "cd755d48-1a0d-41b3-abe7-178686db84fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:16:09.836115 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.836095    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq" (OuterVolumeSpecName: "kube-api-access-grrqq") pod "cd755d48-1a0d-41b3-abe7-178686db84fe" (UID: "cd755d48-1a0d-41b3-abe7-178686db84fe"). InnerVolumeSpecName "kube-api-access-grrqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:16:09.935381 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.935346    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd755d48-1a0d-41b3-abe7-178686db84fe-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:09.935381 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:09.935378    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grrqq\" (UniqueName: \"kubernetes.io/projected/cd755d48-1a0d-41b3-abe7-178686db84fe-kube-api-access-grrqq\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:10.196263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.196166    2550 generic.go:356] "Generic (PLEG): container finished" podID="cd755d48-1a0d-41b3-abe7-178686db84fe" containerID="d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994" exitCode=0
Sep 18 18:16:10.196263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.196215    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"
Sep 18 18:16:10.196263 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.196253    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" event={"ID":"cd755d48-1a0d-41b3-abe7-178686db84fe","Type":"ContainerDied","Data":"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"}
Sep 18 18:16:10.196538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.196310    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc" event={"ID":"cd755d48-1a0d-41b3-abe7-178686db84fe","Type":"ContainerDied","Data":"44352e0c3debf8b10205f2dff9096cf356d23db9211b42686f6ea7fa74e60eb6"}
Sep 18 18:16:10.196538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.196337    2550 scope.go:117] "RemoveContainer" containerID="d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"
Sep 18 18:16:10.209818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.209801    2550 scope.go:117] "RemoveContainer" containerID="d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"
Sep 18 18:16:10.210136 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:16:10.210116    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994\": container with ID starting with d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994 not found: ID does not exist" containerID="d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"
Sep 18 18:16:10.210229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.210146    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994"} err="failed to get container status \"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994\": rpc error: code = NotFound desc = could not find container \"d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994\": container with ID starting with d5cb6d30082fc42f07bb61c4b3c729903fee1b5744e80deb2638f221b4553994 not found: ID does not exist"
Sep 18 18:16:10.218831 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.218808    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:16:10.222730 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.222699    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-89dea9e1-7969c8dc9f-vjxmc"]
Sep 18 18:16:10.706648 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.706615    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd755d48-1a0d-41b3-abe7-178686db84fe" path="/var/lib/kubelet/pods/cd755d48-1a0d-41b3-abe7-178686db84fe/volumes"
Sep 18 18:16:10.707027 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:10.706975    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e48b28e9-0687-4477-a142-cf8ab72744a4" path="/var/lib/kubelet/pods/e48b28e9-0687-4477-a142-cf8ab72744a4/volumes"
Sep 18 18:16:11.588874 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.588839    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:11.589290 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589275    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd755d48-1a0d-41b3-abe7-178686db84fe" containerName="payload-processing"
Sep 18 18:16:11.589356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589291    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd755d48-1a0d-41b3-abe7-178686db84fe" containerName="payload-processing"
Sep 18 18:16:11.589356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589300    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48b28e9-0687-4477-a142-cf8ab72744a4" containerName="revoke-keys"
Sep 18 18:16:11.589356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589306    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48b28e9-0687-4477-a142-cf8ab72744a4" containerName="revoke-keys"
Sep 18 18:16:11.589504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589375    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd755d48-1a0d-41b3-abe7-178686db84fe" containerName="payload-processing"
Sep 18 18:16:11.589504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.589387    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48b28e9-0687-4477-a142-cf8ab72744a4" containerName="revoke-keys"
Sep 18 18:16:11.593413 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.593392    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.596575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.596557    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-94929d65-openshift-default-dockercfg-882fr\""
Sep 18 18:16:11.609235 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.609071    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:11.752072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752033    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752070    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752089    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752176    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752228    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752248    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752313    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752354    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmc2d\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.752575 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.752378    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853618 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853518    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853618 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853561    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853618 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853581    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853625    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853671    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853695    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853734    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853766    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmc2d\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.853911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.853800    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.854264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.854041    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.854264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.854065    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.854376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.854331    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.854432 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.854383    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.854656 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.854517    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.856188 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.856165    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.856418 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.856396    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.862540 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.862513    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.862679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.862660    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmc2d\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d\") pod \"e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") " pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:11.904091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:11.904061    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:12.182664 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:12.182632    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:12.184903 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:16:12.184871    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71263c09_b254_40a7_9904_f6cdad9ca821.slice/crio-d4cb55608de054114da232e715258526535b35b2ed34eba9d10aa43ba86d462f WatchSource:0}: Error finding container d4cb55608de054114da232e715258526535b35b2ed34eba9d10aa43ba86d462f: Status 404 returned error can't find the container with id d4cb55608de054114da232e715258526535b35b2ed34eba9d10aa43ba86d462f
Sep 18 18:16:12.187581 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:12.187545    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:12.187663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:12.187617    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:12.187663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:12.187649    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:12.208664 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:12.208635    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerStarted","Data":"d4cb55608de054114da232e715258526535b35b2ed34eba9d10aa43ba86d462f"}
Sep 18 18:16:13.216514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:13.216478    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerStarted","Data":"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"}
Sep 18 18:16:13.242644 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:13.242590    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podStartSLOduration=2.242574045 podStartE2EDuration="2.242574045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:16:13.240609818 +0000 UTC m=+944.927239330" watchObservedRunningTime="2026-09-18 18:16:13.242574045 +0000 UTC m=+944.929203496"
Sep 18 18:16:13.904589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:13.904549    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:13.910075 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:13.910048    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:14.222004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:14.221891    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:14.222991 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:14.222972    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:19.186644 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:19.186604    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:16:36.980291 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:36.980194    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:16:40.361584 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.361552    2550 generic.go:356] "Generic (PLEG): container finished" podID="71263c09-b254-40a7-9904-f6cdad9ca821" containerID="65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5" exitCode=0
Sep 18 18:16:40.362025 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.361626    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerDied","Data":"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"}
Sep 18 18:16:40.362084 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.362068    2550 scope.go:117] "RemoveContainer" containerID="65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"
Sep 18 18:16:40.363254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.363225    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:40.363348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.363296    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:40.363348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:40.363330    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:16:41.370106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:41.370066    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerStarted","Data":"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"}
Sep 18 18:16:41.904310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:41.904280    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:41.905462 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:41.905431    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:41.905611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:41.905485    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:42.904775 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:42.904739    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:42.905201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:42.904798    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:43.905223 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:43.905182    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:43.905649 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:43.905244    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:44.222878 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:44.222785    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:44.904513 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:44.904472    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:44.904703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:44.904536    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:45.905085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:45.905051    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:45.905474 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:45.905109    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:46.904912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:46.904876    2550 patch_prober.go:28] interesting pod/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 18 18:16:46.905212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:46.905174    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 18 18:16:46.918237 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:46.918211    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:46.918420 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:46.918398    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy" containerID="cri-o://4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e" gracePeriod=30
Sep 18 18:16:52.870304 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.870272    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:52.954094 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954010    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954094 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954049    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954094 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954092    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954122    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmc2d\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954137    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954180    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954212    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954255    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954287    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data\") pod \"71263c09-b254-40a7-9904-f6cdad9ca821\" (UID: \"71263c09-b254-40a7-9904-f6cdad9ca821\") "
Sep 18 18:16:52.954330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954295    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:16:52.954671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954444    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:16:52.954671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954456    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:16:52.954671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954517    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data" (OuterVolumeSpecName: "istio-data") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:16:52.954833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954780    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:52.954833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954798    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:52.954833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954813    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:52.954833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954815    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:16:52.954833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.954826    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:52.956599 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.956566    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d" (OuterVolumeSpecName: "kube-api-access-pmc2d") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "kube-api-access-pmc2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:16:52.956599 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.956592    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token" (OuterVolumeSpecName: "istio-token") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:16:52.956784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.956633    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:16:52.956784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:52.956670    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71263c09-b254-40a7-9904-f6cdad9ca821" (UID: "71263c09-b254-40a7-9904-f6cdad9ca821"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:16:53.056164 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.056130    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71263c09-b254-40a7-9904-f6cdad9ca821-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:53.056164 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.056156    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71263c09-b254-40a7-9904-f6cdad9ca821-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:53.056365 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.056166    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71263c09-b254-40a7-9904-f6cdad9ca821-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:53.056365 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.056187    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:53.056365 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.056196    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmc2d\" (UniqueName: \"kubernetes.io/projected/71263c09-b254-40a7-9904-f6cdad9ca821-kube-api-access-pmc2d\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:16:53.435499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.435462    2550 generic.go:356] "Generic (PLEG): container finished" podID="71263c09-b254-40a7-9904-f6cdad9ca821" containerID="4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e" exitCode=0
Sep 18 18:16:53.435499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.435512    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"
Sep 18 18:16:53.435777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.435545    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerDied","Data":"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"}
Sep 18 18:16:53.435777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.435614    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r" event={"ID":"71263c09-b254-40a7-9904-f6cdad9ca821","Type":"ContainerDied","Data":"d4cb55608de054114da232e715258526535b35b2ed34eba9d10aa43ba86d462f"}
Sep 18 18:16:53.435777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.435642    2550 scope.go:117] "RemoveContainer" containerID="4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"
Sep 18 18:16:53.445051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.445031    2550 scope.go:117] "RemoveContainer" containerID="65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"
Sep 18 18:16:53.454558 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.454535    2550 scope.go:117] "RemoveContainer" containerID="4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"
Sep 18 18:16:53.454861 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:16:53.454841    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e\": container with ID starting with 4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e not found: ID does not exist" containerID="4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"
Sep 18 18:16:53.454924 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.454872    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e"} err="failed to get container status \"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e\": rpc error: code = NotFound desc = could not find container \"4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e\": container with ID starting with 4c2457e508e29b0597f2ab3b355f84a76a9c4494325e17702ed6febce7fb773e not found: ID does not exist"
Sep 18 18:16:53.454924 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.454891    2550 scope.go:117] "RemoveContainer" containerID="65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"
Sep 18 18:16:53.455153 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:16:53.455137    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5\": container with ID starting with 65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5 not found: ID does not exist" containerID="65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"
Sep 18 18:16:53.455212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.455157    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5"} err="failed to get container status \"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5\": rpc error: code = NotFound desc = could not find container \"65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5\": container with ID starting with 65b3b77b05650150c7b82e6e12ca67f85f35a688e2ce329574e5c9d89daf25d5 not found: ID does not exist"
Sep 18 18:16:53.459145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.459122    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:53.465651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.465622    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-94929d65-openshift-default-74b74dcd4-8qx9r"]
Sep 18 18:16:53.781923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.781887    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:16:53.782356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782329    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.782356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782350    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.782556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782374    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.782556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782380    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.782556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782446    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.782556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.782456    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" containerName="istio-proxy"
Sep 18 18:16:53.787217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.787192    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.789502 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.789479    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"premium-w0-37c153-kserve-self-signed-certs\""
Sep 18 18:16:53.795472 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.795443    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:16:53.864380 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864346    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.864380 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864387    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.864654 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864462    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.864654 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864504    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.864654 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864591    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.864654 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.864637    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg2tp\" (UniqueName: \"kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.883089 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.883056    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:16:53.966086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966052    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966098    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966148    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966172    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966276    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966325    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg2tp\" (UniqueName: \"kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966517    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966577    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.966656 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.966618    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.968417 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.968389    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.968789 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.968771    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:53.974603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:53.974578    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg2tp\" (UniqueName: \"kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp\") pod \"premium-w0-37c153-kserve-79bb469d6b-f2j7q\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") " pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:54.097115 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:54.097031    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:16:54.380957 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:54.380913    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:16:54.383851 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:16:54.383819    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4097882b_92af_442c_acf6_a105ac41a828.slice/crio-2ba666a66bd112b436ff7700bfd547b8d28269552c75ebdfadb7fa31fe502b98 WatchSource:0}: Error finding container 2ba666a66bd112b436ff7700bfd547b8d28269552c75ebdfadb7fa31fe502b98: Status 404 returned error can't find the container with id 2ba666a66bd112b436ff7700bfd547b8d28269552c75ebdfadb7fa31fe502b98
Sep 18 18:16:54.444616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:54.444587    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" event={"ID":"4097882b-92af-442c-acf6-a105ac41a828","Type":"ContainerStarted","Data":"2ba666a66bd112b436ff7700bfd547b8d28269552c75ebdfadb7fa31fe502b98"}
Sep 18 18:16:54.465582 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:54.465519    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" podStartSLOduration=1.465503509 podStartE2EDuration="1.465503509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:16:54.463233524 +0000 UTC m=+986.149862999" watchObservedRunningTime="2026-09-18 18:16:54.465503509 +0000 UTC m=+986.152132961"
Sep 18 18:16:54.707601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:54.707515    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71263c09-b254-40a7-9904-f6cdad9ca821" path="/var/lib/kubelet/pods/71263c09-b254-40a7-9904-f6cdad9ca821/volumes"
Sep 18 18:16:55.451079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:16:55.451038    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" event={"ID":"4097882b-92af-442c-acf6-a105ac41a828","Type":"ContainerStarted","Data":"d459eca6a4e5b52fb09e36c820164152b27d1a57f9489f0459302ac245753673"}
Sep 18 18:17:04.097212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:04.097170    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:17:04.097723 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:04.097239    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:17:04.110301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:04.110276    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:17:04.509151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:04.509123    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:17:08.781472 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:08.781438    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:17:24.983277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:24.983232    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:17:51.683040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:17:51.682994    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:18:00.896679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:00.896633    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:18:00.900303 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:00.900280    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:00.903092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:00.903070    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"distinct-w3-039dbc-kserve-self-signed-certs\""
Sep 18 18:18:00.908364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:00.908341    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:18:01.069025 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.068982    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.069212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.069049    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.069212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.069074    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqt6\" (UniqueName: \"kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.069212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.069103    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.069212 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.069199    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.069352 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.069236    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170040    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170089    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170125    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170162    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170177    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqt6\" (UniqueName: \"kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170203    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170593 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170540    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170593 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170538    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.170703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.170621    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.172503 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.172483    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.172804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.172770    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.178154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.178131    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqt6\" (UniqueName: \"kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6\") pod \"distinct-w3-039dbc-kserve-5f68c8df95-4qrbc\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.211119 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.211073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:01.486148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.486050    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:18:01.489114 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:18:01.489081    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c5dad5e_c9ae_49df_89da_9d9392c8c1f1.slice/crio-ea45450fbac5b28c6a99ce5aa7f36cee96f17b84f78950310c2db59be5d01264 WatchSource:0}: Error finding container ea45450fbac5b28c6a99ce5aa7f36cee96f17b84f78950310c2db59be5d01264: Status 404 returned error can't find the container with id ea45450fbac5b28c6a99ce5aa7f36cee96f17b84f78950310c2db59be5d01264
Sep 18 18:18:01.799346 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.799314    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" event={"ID":"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1","Type":"ContainerStarted","Data":"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"}
Sep 18 18:18:01.799521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.799355    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" event={"ID":"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1","Type":"ContainerStarted","Data":"ea45450fbac5b28c6a99ce5aa7f36cee96f17b84f78950310c2db59be5d01264"}
Sep 18 18:18:01.818870 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:01.818810    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" podStartSLOduration=1.818793173 podStartE2EDuration="1.818793173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:18:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:18:01.817332449 +0000 UTC m=+1053.503961920" watchObservedRunningTime="2026-09-18 18:18:01.818793173 +0000 UTC m=+1053.505422624"
Sep 18 18:18:11.211355 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:11.211310    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:11.211764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:11.211390    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:11.224352 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:11.224318    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:11.861978 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:11.861931    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:18:22.795197 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:22.795159    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:18:29.423281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.423247    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:18:29.427076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.427052    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.429655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.429627    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-026961cf\""
Sep 18 18:18:29.429788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.429678    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-026961cf-dockercfg-gd7p9\""
Sep 18 18:18:29.438817 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.438792    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:18:29.545171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.545137    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grn2d\" (UniqueName: \"kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.545349 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.545183    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.645860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.645823    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grn2d\" (UniqueName: \"kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.646072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.645874    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.646887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.646865    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.656265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.656230    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grn2d\" (UniqueName: \"kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d\") pod \"payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") " pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:29.740172 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:29.740081    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:30.012105 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:30.012080    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:18:30.014466 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:18:30.014439    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f94e65f_d458_429f_a49c_bdfa39af661d.slice/crio-d90ebeb1a9b4a0c2d67598107efca94bf386cfec78aeb6eda8e582d6ba363b14 WatchSource:0}: Error finding container d90ebeb1a9b4a0c2d67598107efca94bf386cfec78aeb6eda8e582d6ba363b14: Status 404 returned error can't find the container with id d90ebeb1a9b4a0c2d67598107efca94bf386cfec78aeb6eda8e582d6ba363b14
Sep 18 18:18:30.959137 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:30.959105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" event={"ID":"0f94e65f-d458-429f-a49c-bdfa39af661d","Type":"ContainerStarted","Data":"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"}
Sep 18 18:18:30.959570 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:30.959153    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" event={"ID":"0f94e65f-d458-429f-a49c-bdfa39af661d","Type":"ContainerStarted","Data":"d90ebeb1a9b4a0c2d67598107efca94bf386cfec78aeb6eda8e582d6ba363b14"}
Sep 18 18:18:30.959570 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:30.959185    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:30.974849 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:30.974803    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" podStartSLOduration=1.9747879940000002 podStartE2EDuration="1.974787994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:18:30.973286262 +0000 UTC m=+1082.659915735" watchObservedRunningTime="2026-09-18 18:18:30.974787994 +0000 UTC m=+1082.661417445"
Sep 18 18:18:36.269004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.268960    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:18:36.273052 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.273027    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.275419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.275394    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-039dbc\"/\"distinct-2-w3-039dbc-kserve-self-signed-certs\""
Sep 18 18:18:36.280906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.280884    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:18:36.412832 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.412776    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.413077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.412854    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.413077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.412901    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.413077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.412925    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25fgc\" (UniqueName: \"kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.413077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.413040    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.413077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.413072    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514245 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514202    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514245 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514253    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514284    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514310    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25fgc\" (UniqueName: \"kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514361    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514395    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514658    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514681    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.514778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.514750    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.516669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.516636    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.516858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.516842    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.522099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.522049    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25fgc\" (UniqueName: \"kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc\") pod \"distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") " pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.583628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.583585    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:36.859025 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.858994    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:18:36.860796 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:18:36.860764    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode67a5606_81a2_4e18_9a79_412047699491.slice/crio-29db2111526d1ecd99a4f05f798c97d4c4f95c8ea4d1aee97a6d51af7f8161ce WatchSource:0}: Error finding container 29db2111526d1ecd99a4f05f798c97d4c4f95c8ea4d1aee97a6d51af7f8161ce: Status 404 returned error can't find the container with id 29db2111526d1ecd99a4f05f798c97d4c4f95c8ea4d1aee97a6d51af7f8161ce
Sep 18 18:18:36.992277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.992240    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" event={"ID":"e67a5606-81a2-4e18-9a79-412047699491","Type":"ContainerStarted","Data":"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"}
Sep 18 18:18:36.992473 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:36.992291    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" event={"ID":"e67a5606-81a2-4e18-9a79-412047699491","Type":"ContainerStarted","Data":"29db2111526d1ecd99a4f05f798c97d4c4f95c8ea4d1aee97a6d51af7f8161ce"}
Sep 18 18:18:37.009826 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:37.009762    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" podStartSLOduration=1.009745058 podStartE2EDuration="1.009745058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:18:37.00878013 +0000 UTC m=+1088.695409595" watchObservedRunningTime="2026-09-18 18:18:37.009745058 +0000 UTC m=+1088.696374510"
Sep 18 18:18:41.967020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:41.966977    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:18:46.584349 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:46.584301    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:46.584815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:46.584385    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:46.597051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:46.597013    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:47.056240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:47.056209    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:18:57.836655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.836613    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:18:57.954274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.954233    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"]
Sep 18 18:18:57.958171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.958143    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:18:57.960669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.960638    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:18:57.960669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.960665    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:18:57.968091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:57.968062    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"]
Sep 18 18:18:58.018632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:58.018597    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-026961cf-knzvg\" (UID: \"c69c4e8c-8fa8-4354-b548-b0a23b7edfde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:18:58.119676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:58.119593    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-026961cf-knzvg\" (UID: \"c69c4e8c-8fa8-4354-b548-b0a23b7edfde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:18:58.120370 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:58.120348    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-026961cf-knzvg\" (UID: \"c69c4e8c-8fa8-4354-b548-b0a23b7edfde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:18:58.271648 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:58.271612    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:18:58.550754 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:58.550712    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"]
Sep 18 18:18:58.554357 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:18:58.554328    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69c4e8c_8fa8_4354_b548_b0a23b7edfde.slice/crio-0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f WatchSource:0}: Error finding container 0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f: Status 404 returned error can't find the container with id 0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f
Sep 18 18:18:59.110491 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:59.110395    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg" event={"ID":"c69c4e8c-8fa8-4354-b548-b0a23b7edfde","Type":"ContainerStarted","Data":"17609e3edfe3cb272598b6e5cc60911c31dba8ed01f875f073de8b6814dddf33"}
Sep 18 18:18:59.110491 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:59.110471    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg" event={"ID":"c69c4e8c-8fa8-4354-b548-b0a23b7edfde","Type":"ContainerStarted","Data":"0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f"}
Sep 18 18:18:59.124905 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:18:59.124856    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg" podStartSLOduration=2.124837097 podStartE2EDuration="2.124837097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:18:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:18:59.124345992 +0000 UTC m=+1110.810975477" watchObservedRunningTime="2026-09-18 18:18:59.124837097 +0000 UTC m=+1110.811466548"
Sep 18 18:19:00.119579 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:00.119545    2550 generic.go:356] "Generic (PLEG): container finished" podID="c69c4e8c-8fa8-4354-b548-b0a23b7edfde" containerID="17609e3edfe3cb272598b6e5cc60911c31dba8ed01f875f073de8b6814dddf33" exitCode=0
Sep 18 18:19:00.120007 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:00.119622    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg" event={"ID":"c69c4e8c-8fa8-4354-b548-b0a23b7edfde","Type":"ContainerDied","Data":"17609e3edfe3cb272598b6e5cc60911c31dba8ed01f875f073de8b6814dddf33"}
Sep 18 18:19:01.263668 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:01.263635    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:19:01.353981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:01.353918    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca\") pod \"c69c4e8c-8fa8-4354-b548-b0a23b7edfde\" (UID: \"c69c4e8c-8fa8-4354-b548-b0a23b7edfde\") "
Sep 18 18:19:01.354298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:01.354273    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c69c4e8c-8fa8-4354-b548-b0a23b7edfde" (UID: "c69c4e8c-8fa8-4354-b548-b0a23b7edfde"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:01.454990 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:01.454865    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c69c4e8c-8fa8-4354-b548-b0a23b7edfde-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:02.130860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:02.130830    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"
Sep 18 18:19:02.131047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:02.130866    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg" event={"ID":"c69c4e8c-8fa8-4354-b548-b0a23b7edfde","Type":"ContainerDied","Data":"0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f"}
Sep 18 18:19:02.131047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:02.130913    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d5eab5590b97c6a3cb8e0dc152960ff8c0fa7a96ae6da57ba7558ec5be0105f"
Sep 18 18:19:08.571419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.571379    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:19:08.572009 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.571655    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" podUID="0f94e65f-d458-429f-a49c-bdfa39af661d" containerName="payload-processing" containerID="cri-o://70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc" gracePeriod=30
Sep 18 18:19:08.759621 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.759584    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"]
Sep 18 18:19:08.762450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.762420    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-026961cf-knzvg"]
Sep 18 18:19:08.824667 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.824606    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:19:08.929553 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.929490    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grn2d\" (UniqueName: \"kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d\") pod \"0f94e65f-d458-429f-a49c-bdfa39af661d\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") "
Sep 18 18:19:08.929791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.929638    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume\") pod \"0f94e65f-d458-429f-a49c-bdfa39af661d\" (UID: \"0f94e65f-d458-429f-a49c-bdfa39af661d\") "
Sep 18 18:19:08.930071 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.930043    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f94e65f-d458-429f-a49c-bdfa39af661d" (UID: "0f94e65f-d458-429f-a49c-bdfa39af661d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:08.931906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:08.931882    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d" (OuterVolumeSpecName: "kube-api-access-grn2d") pod "0f94e65f-d458-429f-a49c-bdfa39af661d" (UID: "0f94e65f-d458-429f-a49c-bdfa39af661d"). InnerVolumeSpecName "kube-api-access-grn2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:09.030622 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.030576    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f94e65f-d458-429f-a49c-bdfa39af661d-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.030622 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.030611    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grn2d\" (UniqueName: \"kubernetes.io/projected/0f94e65f-d458-429f-a49c-bdfa39af661d-kube-api-access-grn2d\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:09.168042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.167921    2550 generic.go:356] "Generic (PLEG): container finished" podID="0f94e65f-d458-429f-a49c-bdfa39af661d" containerID="70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc" exitCode=0
Sep 18 18:19:09.168042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.168000    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"
Sep 18 18:19:09.168042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.168009    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" event={"ID":"0f94e65f-d458-429f-a49c-bdfa39af661d","Type":"ContainerDied","Data":"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"}
Sep 18 18:19:09.168255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.168072    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh" event={"ID":"0f94e65f-d458-429f-a49c-bdfa39af661d","Type":"ContainerDied","Data":"d90ebeb1a9b4a0c2d67598107efca94bf386cfec78aeb6eda8e582d6ba363b14"}
Sep 18 18:19:09.168255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.168086    2550 scope.go:117] "RemoveContainer" containerID="70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"
Sep 18 18:19:09.177863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.177846    2550 scope.go:117] "RemoveContainer" containerID="70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"
Sep 18 18:19:09.178176 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:19:09.178152    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc\": container with ID starting with 70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc not found: ID does not exist" containerID="70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"
Sep 18 18:19:09.178268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.178184    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc"} err="failed to get container status \"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc\": rpc error: code = NotFound desc = could not find container \"70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc\": container with ID starting with 70f4cf30ac75c9c12a2269ea400a8c2d548cc8f05e8ab25261b14f9e75c45bfc not found: ID does not exist"
Sep 18 18:19:09.194106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.194060    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:19:09.202316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:09.202290    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-026961cf-9b9fb45d9-blmlh"]
Sep 18 18:19:10.706734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:10.706695    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f94e65f-d458-429f-a49c-bdfa39af661d" path="/var/lib/kubelet/pods/0f94e65f-d458-429f-a49c-bdfa39af661d/volumes"
Sep 18 18:19:10.707155 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:10.707129    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69c4e8c-8fa8-4354-b548-b0a23b7edfde" path="/var/lib/kubelet/pods/c69c4e8c-8fa8-4354-b548-b0a23b7edfde/volumes"
Sep 18 18:19:23.182460 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:23.182415    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:19:45.391605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.391526    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"]
Sep 18 18:19:45.395424 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.395391    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-de7535-29829255-h9rkv"]
Sep 18 18:19:45.422392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.422354    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:19:45.422644 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.422597    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" podUID="5668ea8e-9103-4e3b-b5cd-210043d60414" containerName="payload-processing" containerID="cri-o://44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f" gracePeriod=30
Sep 18 18:19:45.678992 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.678929    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:19:45.796843 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.796812    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vdnp\" (UniqueName: \"kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp\") pod \"5668ea8e-9103-4e3b-b5cd-210043d60414\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") "
Sep 18 18:19:45.797142 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.796859    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume\") pod \"5668ea8e-9103-4e3b-b5cd-210043d60414\" (UID: \"5668ea8e-9103-4e3b-b5cd-210043d60414\") "
Sep 18 18:19:45.797301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.797262    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5668ea8e-9103-4e3b-b5cd-210043d60414" (UID: "5668ea8e-9103-4e3b-b5cd-210043d60414"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:45.799193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.799167    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp" (OuterVolumeSpecName: "kube-api-access-5vdnp") pod "5668ea8e-9103-4e3b-b5cd-210043d60414" (UID: "5668ea8e-9103-4e3b-b5cd-210043d60414"). InnerVolumeSpecName "kube-api-access-5vdnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:45.898281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.898246    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vdnp\" (UniqueName: \"kubernetes.io/projected/5668ea8e-9103-4e3b-b5cd-210043d60414-kube-api-access-5vdnp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:45.898281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:45.898279    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5668ea8e-9103-4e3b-b5cd-210043d60414-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:46.385624 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.385587    2550 generic.go:356] "Generic (PLEG): container finished" podID="5668ea8e-9103-4e3b-b5cd-210043d60414" containerID="44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f" exitCode=0
Sep 18 18:19:46.385803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.385636    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"
Sep 18 18:19:46.385803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.385638    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" event={"ID":"5668ea8e-9103-4e3b-b5cd-210043d60414","Type":"ContainerDied","Data":"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"}
Sep 18 18:19:46.385803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.385767    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6" event={"ID":"5668ea8e-9103-4e3b-b5cd-210043d60414","Type":"ContainerDied","Data":"4480937d8b84188fd7bdb444c415e345d61c842c9427b59aaf4d2d8ea96507a0"}
Sep 18 18:19:46.385803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.385799    2550 scope.go:117] "RemoveContainer" containerID="44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"
Sep 18 18:19:46.396018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.395998    2550 scope.go:117] "RemoveContainer" containerID="44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"
Sep 18 18:19:46.396343 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:19:46.396320    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f\": container with ID starting with 44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f not found: ID does not exist" containerID="44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"
Sep 18 18:19:46.396390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.396352    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f"} err="failed to get container status \"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f\": rpc error: code = NotFound desc = could not find container \"44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f\": container with ID starting with 44e4e32cb62452e4dea2fec6838861b1d8e6f13953083cc881eb6872e5ecda3f not found: ID does not exist"
Sep 18 18:19:46.407587 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.407560    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:19:46.411485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.411463    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-de7535-6dcf799cf-mxln6"]
Sep 18 18:19:46.707050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.706960    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5668ea8e-9103-4e3b-b5cd-210043d60414" path="/var/lib/kubelet/pods/5668ea8e-9103-4e3b-b5cd-210043d60414/volumes"
Sep 18 18:19:46.707277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:46.707263    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1" path="/var/lib/kubelet/pods/b1b1a61c-6640-4cfb-b6b3-6ede7e2a0af1/volumes"
Sep 18 18:19:52.879555 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:52.879523    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:19:52.879976 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:52.879742    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy" containerID="cri-o://af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70" gracePeriod=30
Sep 18 18:19:53.206896 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.206811    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"]
Sep 18 18:19:53.207223 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207210    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69c4e8c-8fa8-4354-b548-b0a23b7edfde" containerName="revoke-keys"
Sep 18 18:19:53.207274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207225    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69c4e8c-8fa8-4354-b548-b0a23b7edfde" containerName="revoke-keys"
Sep 18 18:19:53.207274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207239    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f94e65f-d458-429f-a49c-bdfa39af661d" containerName="payload-processing"
Sep 18 18:19:53.207274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207246    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f94e65f-d458-429f-a49c-bdfa39af661d" containerName="payload-processing"
Sep 18 18:19:53.207274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207262    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5668ea8e-9103-4e3b-b5cd-210043d60414" containerName="payload-processing"
Sep 18 18:19:53.207274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207268    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5668ea8e-9103-4e3b-b5cd-210043d60414" containerName="payload-processing"
Sep 18 18:19:53.207430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207330    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f94e65f-d458-429f-a49c-bdfa39af661d" containerName="payload-processing"
Sep 18 18:19:53.207430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207339    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5668ea8e-9103-4e3b-b5cd-210043d60414" containerName="payload-processing"
Sep 18 18:19:53.207430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.207345    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69c4e8c-8fa8-4354-b548-b0a23b7edfde" containerName="revoke-keys"
Sep 18 18:19:53.210322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.210303    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:53.212558 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.212532    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:19:53.212658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.212627    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:19:53.217855 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.217817    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"]
Sep 18 18:19:53.271501 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.271452    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck\" (UID: \"03821012-f64d-4215-baa7-53f18390806f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:53.373088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.373032    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck\" (UID: \"03821012-f64d-4215-baa7-53f18390806f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:53.373696 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.373673    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck\" (UID: \"03821012-f64d-4215-baa7-53f18390806f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:53.522608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.522578    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:53.803806 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:53.803778    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"]
Sep 18 18:19:53.805638 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:19:53.805602    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03821012_f64d_4215_baa7_53f18390806f.slice/crio-611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd WatchSource:0}: Error finding container 611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd: Status 404 returned error can't find the container with id 611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd
Sep 18 18:19:54.436061 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:54.435968    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck" event={"ID":"03821012-f64d-4215-baa7-53f18390806f","Type":"ContainerStarted","Data":"8599e27e1c12cd1ee09333bacc9fe68ea6a64de95c22fdb308329c404bf97f15"}
Sep 18 18:19:54.436061 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:54.436035    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck" event={"ID":"03821012-f64d-4215-baa7-53f18390806f","Type":"ContainerStarted","Data":"611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd"}
Sep 18 18:19:54.452288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:54.452236    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck" podStartSLOduration=1.452220701 podStartE2EDuration="1.452220701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:54.449805324 +0000 UTC m=+1166.136434814" watchObservedRunningTime="2026-09-18 18:19:54.452220701 +0000 UTC m=+1166.138850153"
Sep 18 18:19:55.444150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:55.444110    2550 generic.go:356] "Generic (PLEG): container finished" podID="03821012-f64d-4215-baa7-53f18390806f" containerID="8599e27e1c12cd1ee09333bacc9fe68ea6a64de95c22fdb308329c404bf97f15" exitCode=0
Sep 18 18:19:55.444617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:55.444193    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck" event={"ID":"03821012-f64d-4215-baa7-53f18390806f","Type":"ContainerDied","Data":"8599e27e1c12cd1ee09333bacc9fe68ea6a64de95c22fdb308329c404bf97f15"}
Sep 18 18:19:56.587314 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:56.587285    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:56.708140 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:56.708055    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca\") pod \"03821012-f64d-4215-baa7-53f18390806f\" (UID: \"03821012-f64d-4215-baa7-53f18390806f\") "
Sep 18 18:19:56.708370 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:56.708348    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03821012-f64d-4215-baa7-53f18390806f" (UID: "03821012-f64d-4215-baa7-53f18390806f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:56.809587 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:56.809540    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03821012-f64d-4215-baa7-53f18390806f-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:57.455814 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.455784    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"
Sep 18 18:19:57.456060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.455821    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck" event={"ID":"03821012-f64d-4215-baa7-53f18390806f","Type":"ContainerDied","Data":"611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd"}
Sep 18 18:19:57.456060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.455867    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="611547c1868a4c48e8a8b2a6e370e0360be615fecef887cb75e6b5641c802bfd"
Sep 18 18:19:57.829778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.829745    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:19:57.830193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.830152    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03821012-f64d-4215-baa7-53f18390806f" containerName="revoke-keys"
Sep 18 18:19:57.830193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.830164    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="03821012-f64d-4215-baa7-53f18390806f" containerName="revoke-keys"
Sep 18 18:19:57.830269 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.830227    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="03821012-f64d-4215-baa7-53f18390806f" containerName="revoke-keys"
Sep 18 18:19:57.833310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.833292    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:57.836082 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.836065    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-1f7000\""
Sep 18 18:19:57.836148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.836086    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-1f7000-dockercfg-42dv6\""
Sep 18 18:19:57.844112 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.844084    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:19:57.871945 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.871900    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:19:57.875326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.875310    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:57.893336 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.893309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:19:57.920258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.920227    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:57.920396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:57.920321    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428th\" (UniqueName: \"kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.021567 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.021524    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428th\" (UniqueName: \"kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.021757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.021593    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxxq4\" (UniqueName: \"kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.021757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.021620    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.021757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.021708    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.022412 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.022394    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.031522 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.031484    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-428th\" (UniqueName: \"kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th\") pod \"payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.122769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.122737    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxxq4\" (UniqueName: \"kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.122911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.122774    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.123638 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.123612    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.128431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.128410    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:19:58.131372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.131350    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxxq4\" (UniqueName: \"kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4\") pod \"payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.144918 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.144891    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:58.186482 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.186444    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:58.224156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224119    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224169    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224204    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224229    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224267    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224291    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224377    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5shqr\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224423    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224462    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data\") pod \"d96f900b-e64c-464d-8a37-294bcb6fafbc\" (UID: \"d96f900b-e64c-464d-8a37-294bcb6fafbc\") "
Sep 18 18:19:58.224632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224569    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:19:58.224856 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224786    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:58.225030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.224993    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96f900b-e64c-464d-8a37-294bcb6fafbc-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.225030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.225022    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.225324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.225278    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:58.225607 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.225580    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:58.225695 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.225649    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:58.227797 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.227775    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr" (OuterVolumeSpecName: "kube-api-access-5shqr") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "kube-api-access-5shqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:58.227887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.227798    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:19:58.227887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.227828    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:19:58.228084 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.227898    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d96f900b-e64c-464d-8a37-294bcb6fafbc" (UID: "d96f900b-e64c-464d-8a37-294bcb6fafbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:19:58.325611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325576    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5shqr\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-kube-api-access-5shqr\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325605    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325616    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325624    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325632    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325640    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96f900b-e64c-464d-8a37-294bcb6fafbc-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.325850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.325648    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96f900b-e64c-464d-8a37-294bcb6fafbc-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:19:58.425473 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.425446    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:19:58.427670 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:19:58.427638    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8fce21_6f3f_4cd1_a74d_fefe4109be87.slice/crio-e3b12b7458d42edb62c54cb3d7c1a246f7a1acbe75b9ea6f4fac323623bf0284 WatchSource:0}: Error finding container e3b12b7458d42edb62c54cb3d7c1a246f7a1acbe75b9ea6f4fac323623bf0284: Status 404 returned error can't find the container with id e3b12b7458d42edb62c54cb3d7c1a246f7a1acbe75b9ea6f4fac323623bf0284
Sep 18 18:19:58.462311 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.462281    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" event={"ID":"bf8fce21-6f3f-4cd1-a74d-fefe4109be87","Type":"ContainerStarted","Data":"e3b12b7458d42edb62c54cb3d7c1a246f7a1acbe75b9ea6f4fac323623bf0284"}
Sep 18 18:19:58.464069 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.464040    2550 generic.go:356] "Generic (PLEG): container finished" podID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerID="af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70" exitCode=0
Sep 18 18:19:58.464187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.464098    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"
Sep 18 18:19:58.464187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.464121    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerDied","Data":"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"}
Sep 18 18:19:58.464187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.464183    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62" event={"ID":"d96f900b-e64c-464d-8a37-294bcb6fafbc","Type":"ContainerDied","Data":"acd72c662fc0524a9df2f30ecdc2a2250f1908d4ee619ded6ceef0998dd9b85f"}
Sep 18 18:19:58.464351 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.464218    2550 scope.go:117] "RemoveContainer" containerID="af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"
Sep 18 18:19:58.473338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.473294    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:19:58.476000 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.475968    2550 scope.go:117] "RemoveContainer" containerID="aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"
Sep 18 18:19:58.477125 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:19:58.477066    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb02a275_aba3_46ff_bdea_480102ada8b0.slice/crio-a253c2b1e80a5b236e5b06cd3fec3651dece5cfd9786d8e47b1521ed495b51d0 WatchSource:0}: Error finding container a253c2b1e80a5b236e5b06cd3fec3651dece5cfd9786d8e47b1521ed495b51d0: Status 404 returned error can't find the container with id a253c2b1e80a5b236e5b06cd3fec3651dece5cfd9786d8e47b1521ed495b51d0
Sep 18 18:19:58.485848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.485825    2550 scope.go:117] "RemoveContainer" containerID="af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"
Sep 18 18:19:58.486284 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:19:58.486258    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70\": container with ID starting with af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70 not found: ID does not exist" containerID="af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"
Sep 18 18:19:58.486347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.486296    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70"} err="failed to get container status \"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70\": rpc error: code = NotFound desc = could not find container \"af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70\": container with ID starting with af9b924c899c1ab1f6e22cb7b449fc3ffc574ec19abfeb2791f8a93221d55b70 not found: ID does not exist"
Sep 18 18:19:58.486347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.486321    2550 scope.go:117] "RemoveContainer" containerID="aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"
Sep 18 18:19:58.486655 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:19:58.486608    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea\": container with ID starting with aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea not found: ID does not exist" containerID="aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"
Sep 18 18:19:58.486655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.486634    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea"} err="failed to get container status \"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea\": rpc error: code = NotFound desc = could not find container \"aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea\": container with ID starting with aef4c743b93626eb5b8a771f65701ef3aa61817ea0cb7b507023484634ce9eea not found: ID does not exist"
Sep 18 18:19:58.488305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.488285    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:19:58.494956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.494920    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-de7535-openshift-default-69b96cc448-mgt62"]
Sep 18 18:19:58.706665 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:58.706570    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" path="/var/lib/kubelet/pods/d96f900b-e64c-464d-8a37-294bcb6fafbc/volumes"
Sep 18 18:19:59.472764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.472726    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" event={"ID":"bf8fce21-6f3f-4cd1-a74d-fefe4109be87","Type":"ContainerStarted","Data":"a26a26f9a6215fe65f625901dc6da18056c8f82c0d4e8effe22c69c28629305c"}
Sep 18 18:19:59.473288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.472907    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:19:59.475589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.475559    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" event={"ID":"bb02a275-aba3-46ff-bdea-480102ada8b0","Type":"ContainerStarted","Data":"7b0ca4685c6e19ed3281d4881cefbb480a6fbf1c50bf062511c978f55b66b301"}
Sep 18 18:19:59.475742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.475610    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" event={"ID":"bb02a275-aba3-46ff-bdea-480102ada8b0","Type":"ContainerStarted","Data":"a253c2b1e80a5b236e5b06cd3fec3651dece5cfd9786d8e47b1521ed495b51d0"}
Sep 18 18:19:59.475742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.475712    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:19:59.488297 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.488243    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" podStartSLOduration=2.488223448 podStartE2EDuration="2.488223448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:59.487798177 +0000 UTC m=+1171.174427628" watchObservedRunningTime="2026-09-18 18:19:59.488223448 +0000 UTC m=+1171.174852901"
Sep 18 18:19:59.507605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:19:59.507541    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" podStartSLOduration=2.5075188390000003 podStartE2EDuration="2.507518839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:19:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:19:59.507046132 +0000 UTC m=+1171.193675606" watchObservedRunningTime="2026-09-18 18:19:59.507518839 +0000 UTC m=+1171.194148291"
Sep 18 18:20:03.921131 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:03.921097    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:20:03.921617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:03.921364    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" podUID="e28808bd-e43c-4f85-98eb-ea8f3818149c" containerName="payload-pre-processing" containerID="cri-o://26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a" gracePeriod=30
Sep 18 18:20:04.102360 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.102324    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"]
Sep 18 18:20:04.104446 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.104399    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-bcf02f-gvcck"]
Sep 18 18:20:04.174137 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.174073    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:20:04.184374 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.184349    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:20:04.287297 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.287242    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume\") pod \"e28808bd-e43c-4f85-98eb-ea8f3818149c\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") "
Sep 18 18:20:04.287505 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.287395    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jpb29\" (UniqueName: \"kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29\") pod \"e28808bd-e43c-4f85-98eb-ea8f3818149c\" (UID: \"e28808bd-e43c-4f85-98eb-ea8f3818149c\") "
Sep 18 18:20:04.287689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.287661    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e28808bd-e43c-4f85-98eb-ea8f3818149c" (UID: "e28808bd-e43c-4f85-98eb-ea8f3818149c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:20:04.289634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.289599    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29" (OuterVolumeSpecName: "kube-api-access-jpb29") pod "e28808bd-e43c-4f85-98eb-ea8f3818149c" (UID: "e28808bd-e43c-4f85-98eb-ea8f3818149c"). InnerVolumeSpecName "kube-api-access-jpb29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:04.388357 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.388317    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e28808bd-e43c-4f85-98eb-ea8f3818149c-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:04.388357 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.388347    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jpb29\" (UniqueName: \"kubernetes.io/projected/e28808bd-e43c-4f85-98eb-ea8f3818149c-kube-api-access-jpb29\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:04.508492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.508461    2550 generic.go:356] "Generic (PLEG): container finished" podID="e28808bd-e43c-4f85-98eb-ea8f3818149c" containerID="26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a" exitCode=0
Sep 18 18:20:04.508633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.508508    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"
Sep 18 18:20:04.508633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.508548    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" event={"ID":"e28808bd-e43c-4f85-98eb-ea8f3818149c","Type":"ContainerDied","Data":"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"}
Sep 18 18:20:04.508633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.508604    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm" event={"ID":"e28808bd-e43c-4f85-98eb-ea8f3818149c","Type":"ContainerDied","Data":"c6d321098706fba17f449f152ec2a2ecb9e8142ddb7b7394af53e9ea1fffc2ea"}
Sep 18 18:20:04.508750 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.508633    2550 scope.go:117] "RemoveContainer" containerID="26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"
Sep 18 18:20:04.518882 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.518858    2550 scope.go:117] "RemoveContainer" containerID="26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"
Sep 18 18:20:04.519228 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:20:04.519205    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a\": container with ID starting with 26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a not found: ID does not exist" containerID="26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"
Sep 18 18:20:04.519321 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.519239    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a"} err="failed to get container status \"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a\": rpc error: code = NotFound desc = could not find container \"26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a\": container with ID starting with 26f554c538e6092b8ddbb9dc0c30ea5a5a2b09ed7f2289a329b9396098eaa86a not found: ID does not exist"
Sep 18 18:20:04.536303 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.536277    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:20:04.538147 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.538122    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-bcf02f-589d4b798b-zgfhm"]
Sep 18 18:20:04.708535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.708496    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03821012-f64d-4215-baa7-53f18390806f" path="/var/lib/kubelet/pods/03821012-f64d-4215-baa7-53f18390806f/volumes"
Sep 18 18:20:04.708971 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:04.708953    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28808bd-e43c-4f85-98eb-ea8f3818149c" path="/var/lib/kubelet/pods/e28808bd-e43c-4f85-98eb-ea8f3818149c/volumes"
Sep 18 18:20:10.484556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:10.484514    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:20:10.485002 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:10.484594    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:20:11.040510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.040483    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:20:11.040721 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.040691    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy" containerID="cri-o://36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb" gracePeriod=30
Sep 18 18:20:11.176360 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176326    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:20:11.176766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176748    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:11.176848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176768    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:11.176848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176797    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28808bd-e43c-4f85-98eb-ea8f3818149c" containerName="payload-pre-processing"
Sep 18 18:20:11.176848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176806    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28808bd-e43c-4f85-98eb-ea8f3818149c" containerName="payload-pre-processing"
Sep 18 18:20:11.177041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176911    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:11.177041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.176926    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28808bd-e43c-4f85-98eb-ea8f3818149c" containerName="payload-pre-processing"
Sep 18 18:20:11.181269 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.181249    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.183644 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.183624    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"distinct-2-w0-37c153-kserve-self-signed-certs\""
Sep 18 18:20:11.189162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.188869    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:20:11.251647 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251618    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.251756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251673    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.251756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251698    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l92xp\" (UniqueName: \"kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.251756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251727    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.251886 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251766    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.251886 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.251786    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352383 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352321    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352383 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352354    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352383 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352378    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352508    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352552    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l92xp\" (UniqueName: \"kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352597    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352664    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352741    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.352925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.352899    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.354605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.354587    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.354868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.354852    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.360982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.360958    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l92xp\" (UniqueName: \"kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp\") pod \"distinct-2-w0-37c153-kserve-7cb65547f4-jknk7\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") " pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.491440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.491411    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:11.763390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:11.763365    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:20:11.765294 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:11.765265    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47f344b7_76cf_49b9_9bec_ac155eb5c167.slice/crio-fdb4cc911ba3a83d10e75b3e93a5ace3ce93ab424c281ddf9607791ce17be9c1 WatchSource:0}: Error finding container fdb4cc911ba3a83d10e75b3e93a5ace3ce93ab424c281ddf9607791ce17be9c1: Status 404 returned error can't find the container with id fdb4cc911ba3a83d10e75b3e93a5ace3ce93ab424c281ddf9607791ce17be9c1
Sep 18 18:20:12.558439 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:12.558395    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" event={"ID":"47f344b7-76cf-49b9-9bec-ac155eb5c167","Type":"ContainerStarted","Data":"dbc628ae33f4fbfc87eb52f912d46eca5ea0419122cd84d870cad0e19f42aa32"}
Sep 18 18:20:12.558979 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:12.558447    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" event={"ID":"47f344b7-76cf-49b9-9bec-ac155eb5c167","Type":"ContainerStarted","Data":"fdb4cc911ba3a83d10e75b3e93a5ace3ce93ab424c281ddf9607791ce17be9c1"}
Sep 18 18:20:12.576857 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:12.576813    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" podStartSLOduration=1.576797945 podStartE2EDuration="1.576797945s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:12.575703516 +0000 UTC m=+1184.262333012" watchObservedRunningTime="2026-09-18 18:20:12.576797945 +0000 UTC m=+1184.263427398"
Sep 18 18:20:16.293466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.293444    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:20:16.396911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.396835    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.396911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.396891    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.396962    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397011    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wjw4\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397048    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397072    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397142    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397178    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397223    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs\") pod \"9471350e-1cae-41bc-968e-d55dda73e6b3\" (UID: \"9471350e-1cae-41bc-968e-d55dda73e6b3\") "
Sep 18 18:20:16.397663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397324    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:20:16.397663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397436    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:20:16.397663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397554    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:20:16.397663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397559    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.397663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397593    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.397889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397750    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:20:16.397889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.397770    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:20:16.399378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.399354    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4" (OuterVolumeSpecName: "kube-api-access-2wjw4") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "kube-api-access-2wjw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:16.399378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.399371    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:20:16.399583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.399565    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:20:16.399638 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.399584    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9471350e-1cae-41bc-968e-d55dda73e6b3" (UID: "9471350e-1cae-41bc-968e-d55dda73e6b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:20:16.499107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499073    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wjw4\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-kube-api-access-2wjw4\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499100    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499109    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499118    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9471350e-1cae-41bc-968e-d55dda73e6b3-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499127    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499135    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.499415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.499142    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9471350e-1cae-41bc-968e-d55dda73e6b3-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:20:16.581101 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.581064    2550 generic.go:356] "Generic (PLEG): container finished" podID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerID="36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb" exitCode=0
Sep 18 18:20:16.581270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.581129    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"
Sep 18 18:20:16.581270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.581149    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerDied","Data":"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"}
Sep 18 18:20:16.581270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.581235    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb" event={"ID":"9471350e-1cae-41bc-968e-d55dda73e6b3","Type":"ContainerDied","Data":"81ba9b2d8d7dc6eebca850994c51b4c63dfd0c58106e2fad47fd9ec9eb9ec310"}
Sep 18 18:20:16.581270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.581259    2550 scope.go:117] "RemoveContainer" containerID="36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"
Sep 18 18:20:16.591675 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.591657    2550 scope.go:117] "RemoveContainer" containerID="e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"
Sep 18 18:20:16.600390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.600373    2550 scope.go:117] "RemoveContainer" containerID="36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"
Sep 18 18:20:16.600669 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:20:16.600650    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb\": container with ID starting with 36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb not found: ID does not exist" containerID="36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"
Sep 18 18:20:16.600724 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.600679    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb"} err="failed to get container status \"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb\": rpc error: code = NotFound desc = could not find container \"36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb\": container with ID starting with 36c08c0253faf15bc261c40159e318c6e2fec2adf4bf7b4ad943fb9adda365eb not found: ID does not exist"
Sep 18 18:20:16.600724 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.600694    2550 scope.go:117] "RemoveContainer" containerID="e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"
Sep 18 18:20:16.601013 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:20:16.600920    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48\": container with ID starting with e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48 not found: ID does not exist" containerID="e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"
Sep 18 18:20:16.601013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.600967    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48"} err="failed to get container status \"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48\": rpc error: code = NotFound desc = could not find container \"e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48\": container with ID starting with e80000c3fc3b9ca31b2c9caffb0d0da8b5e3a3b7f3fa3118c64b8fc6019c1e48 not found: ID does not exist"
Sep 18 18:20:16.603666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.603645    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:20:16.609369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.609349    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bcf02f-openshift-default-67c665cd9d-vfxvb"]
Sep 18 18:20:16.706281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:16.706228    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" path="/var/lib/kubelet/pods/9471350e-1cae-41bc-968e-d55dda73e6b3/volumes"
Sep 18 18:20:19.360782 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.360751    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:20:19.361152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361105    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.361152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361116    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.361152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361129    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:19.361152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361134    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:19.361279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361201    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.361279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361209    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.361279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.361219    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96f900b-e64c-464d-8a37-294bcb6fafbc" containerName="istio-proxy"
Sep 18 18:20:19.364378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.364360    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.368594 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.368573    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-011541-dockercfg-zwdvc\""
Sep 18 18:20:19.368729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.368616    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-011541\""
Sep 18 18:20:19.374371 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.374346    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:20:19.423762 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.423742    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww6vc\" (UniqueName: \"kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.423860 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.423823    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.524183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.524159    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.524292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.524196    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww6vc\" (UniqueName: \"kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.524827 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.524806    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.531717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.531692    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww6vc\" (UniqueName: \"kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc\") pod \"payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.589981 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.589954    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:20:19.590364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.590348    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.590443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.590368    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9471350e-1cae-41bc-968e-d55dda73e6b3" containerName="istio-proxy"
Sep 18 18:20:19.594729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.594711    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.597337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.597318    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 18:20:19.597337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.597329    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6fa1a2a2-openshift-default-dockercfg-9dqcv\""
Sep 18 18:20:19.604049 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.604031    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:20:19.625298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625188    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msmlz\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625221    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625251    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625294    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625398    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625435    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625466    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.625598 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.625505    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.675928 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.675905    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:19.726279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.726251    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msmlz\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.726399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.726306    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.726399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.726338    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727005 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.726979    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727117 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727029    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727117 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727061    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727117 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727109    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727269 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727146    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727269 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727176    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727611    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727894    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.727996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.727924    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.728209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.728142    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.728409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.728390    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.731179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.731155    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.731322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.731196    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.741338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.741305    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.743226 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.743187    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msmlz\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz\") pod \"e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") " pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.906392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.906320    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:19.947391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:19.947363    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:20:19.949918 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:19.949893    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca105863_19e3_4dd4_b16e_6e82e0fe8f42.slice/crio-8b27230ff2200c3098be9b3a426e104071e218b2b10b5284374eceee457fa793 WatchSource:0}: Error finding container 8b27230ff2200c3098be9b3a426e104071e218b2b10b5284374eceee457fa793: Status 404 returned error can't find the container with id 8b27230ff2200c3098be9b3a426e104071e218b2b10b5284374eceee457fa793
Sep 18 18:20:20.185335 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.185309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:20:20.187350 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:20.187325    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e29416_e22e_483f_9647_233a72bee579.slice/crio-7c88e0fca27ac1664a7f91cd0839a80f80f0b5f3c6105584a98b363ca5ebc8ec WatchSource:0}: Error finding container 7c88e0fca27ac1664a7f91cd0839a80f80f0b5f3c6105584a98b363ca5ebc8ec: Status 404 returned error can't find the container with id 7c88e0fca27ac1664a7f91cd0839a80f80f0b5f3c6105584a98b363ca5ebc8ec
Sep 18 18:20:20.189631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.189595    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:20.189726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.189663    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:20.189726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.189690    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:20.605827 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.605783    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" event={"ID":"ca105863-19e3-4dd4-b16e-6e82e0fe8f42","Type":"ContainerStarted","Data":"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"}
Sep 18 18:20:20.606275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.605858    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" event={"ID":"ca105863-19e3-4dd4-b16e-6e82e0fe8f42","Type":"ContainerStarted","Data":"8b27230ff2200c3098be9b3a426e104071e218b2b10b5284374eceee457fa793"}
Sep 18 18:20:20.606275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.605891    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:20.607777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.607742    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerStarted","Data":"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"}
Sep 18 18:20:20.607898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.607812    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerStarted","Data":"7c88e0fca27ac1664a7f91cd0839a80f80f0b5f3c6105584a98b363ca5ebc8ec"}
Sep 18 18:20:20.648907 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.648865    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" podStartSLOduration=1.648851041 podStartE2EDuration="1.648851041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:20.647825716 +0000 UTC m=+1192.334455190" watchObservedRunningTime="2026-09-18 18:20:20.648851041 +0000 UTC m=+1192.335480488"
Sep 18 18:20:20.672087 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.672036    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podStartSLOduration=1.672023324 podStartE2EDuration="1.672023324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:20.671059812 +0000 UTC m=+1192.357689263" watchObservedRunningTime="2026-09-18 18:20:20.672023324 +0000 UTC m=+1192.358652774"
Sep 18 18:20:20.906894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:20.906828    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:21.492425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:21.492392    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:21.492586 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:21.492435    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:21.504532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:21.504504    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:21.625514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:21.625493    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:20:21.912054 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:21.912019    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:22.620903 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:22.620868    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:22.621603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:22.621587    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:20:27.857686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.857650    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:20:27.861537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.861504    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:27.864030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.863992    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6fa1a2a2-dockercfg-lgntz\""
Sep 18 18:20:27.864192 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.864043    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6fa1a2a2\""
Sep 18 18:20:27.871714 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.871692    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:20:27.997443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.997404    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7hjz\" (UniqueName: \"kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:27.997642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:27.997474    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.098615 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.098581    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7hjz\" (UniqueName: \"kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.098783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.098641    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.099325 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.099307    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.106623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.106602    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7hjz\" (UniqueName: \"kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz\") pod \"payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.174588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.174529    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.447710 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.447677    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:20:28.449388 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:28.449361    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6107d2c_5a40_4814_aa37_cfb9277b831b.slice/crio-b05d3bee0bc24e2b30888c304b0edceda089cd6cdd479cf4a65f48441a7142e1 WatchSource:0}: Error finding container b05d3bee0bc24e2b30888c304b0edceda089cd6cdd479cf4a65f48441a7142e1: Status 404 returned error can't find the container with id b05d3bee0bc24e2b30888c304b0edceda089cd6cdd479cf4a65f48441a7142e1
Sep 18 18:20:28.654427 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.654375    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" event={"ID":"f6107d2c-5a40-4814-aa37-cfb9277b831b","Type":"ContainerStarted","Data":"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"}
Sep 18 18:20:28.654615 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.654445    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" event={"ID":"f6107d2c-5a40-4814-aa37-cfb9277b831b","Type":"ContainerStarted","Data":"b05d3bee0bc24e2b30888c304b0edceda089cd6cdd479cf4a65f48441a7142e1"}
Sep 18 18:20:28.654615 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.654477    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:28.671719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:28.671661    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" podStartSLOduration=1.671645206 podStartE2EDuration="1.671645206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:28.669802852 +0000 UTC m=+1200.356432355" watchObservedRunningTime="2026-09-18 18:20:28.671645206 +0000 UTC m=+1200.358274706"
Sep 18 18:20:30.744382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:30.744352    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:20:30.746300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:30.746277    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:20:31.616503 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:31.616472    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:20:39.663086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:39.663058    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:20:46.083666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.083634    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:20:46.088755 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.088735    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.091271 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.091247    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-99602c-openshift-default-dockercfg-xkmj2\""
Sep 18 18:20:46.098549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.098522    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:20:46.257274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257280    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257306    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257409    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257499 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257494    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257681 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257533    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257681 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257576    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhq42\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257681 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257603    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.257681 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.257660    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.358653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358559    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.358653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358596    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.358653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358625    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhq42\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.358653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358648    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358689    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358731    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358757    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358781    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.358833    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.359179    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.359239    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359410 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.359331    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359410 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.359387    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.359600 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.359575    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.361542 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.361496    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.361657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.361552    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.367026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.366995    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.367191 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.367172    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhq42\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42\") pod \"e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") " pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.400183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.400145    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:46.678393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.678363    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:20:46.680448 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:46.680420    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05790c99_3826_4f5e_add1_e82e5a96f1dc.slice/crio-cd5c62402c4f13188655d5d6535ac74bac87e89d65476f4220ea83f93efc57da WatchSource:0}: Error finding container cd5c62402c4f13188655d5d6535ac74bac87e89d65476f4220ea83f93efc57da: Status 404 returned error can't find the container with id cd5c62402c4f13188655d5d6535ac74bac87e89d65476f4220ea83f93efc57da
Sep 18 18:20:46.683042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.682996    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:46.683136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.683100    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:46.683185 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.683144    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:20:46.753816 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.753787    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerStarted","Data":"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"}
Sep 18 18:20:46.753932 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.753852    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerStarted","Data":"cd5c62402c4f13188655d5d6535ac74bac87e89d65476f4220ea83f93efc57da"}
Sep 18 18:20:46.777060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:46.777013    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podStartSLOduration=0.776996998 podStartE2EDuration="776.996998ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:46.77593126 +0000 UTC m=+1218.462560735" watchObservedRunningTime="2026-09-18 18:20:46.776996998 +0000 UTC m=+1218.463626450"
Sep 18 18:20:47.400814 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:47.400776    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:47.405974 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:47.405919    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:47.759300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:47.759272    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:47.760252 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:47.760232    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:20:49.291703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:49.291668    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:20:50.858426 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:50.858394    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:20:50.862039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:50.862016    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:50.864424 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:50.864388    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-99602c\""
Sep 18 18:20:50.864531 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:50.864504    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-99602c-dockercfg-nnfjw\""
Sep 18 18:20:50.872044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:50.872023    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:20:51.007355 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.007329    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nx64\" (UniqueName: \"kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.007537 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.007381    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.109050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.108978    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nx64\" (UniqueName: \"kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.109050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.109025    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.109690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.109671    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.116924 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.116903    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nx64\" (UniqueName: \"kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64\") pod \"payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.173159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.173116    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:20:51.463643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.463610    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:20:51.467089 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:20:51.466209    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fd1729_6f5b_4fb4_9240_e513cabed3b6.slice/crio-b535c3fcb7121266f2885e6337a8f74a43ef6588f0d6992788c69b782ecf9351 WatchSource:0}: Error finding container b535c3fcb7121266f2885e6337a8f74a43ef6588f0d6992788c69b782ecf9351: Status 404 returned error can't find the container with id b535c3fcb7121266f2885e6337a8f74a43ef6588f0d6992788c69b782ecf9351
Sep 18 18:20:51.783170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.783125    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" event={"ID":"e9fd1729-6f5b-4fb4-9240-e513cabed3b6","Type":"ContainerStarted","Data":"e430b06be60adc94ef2ebb81a76e806af0fc63eacce6d806b42b83e34d416048"}
Sep 18 18:20:51.783392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.783217    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" event={"ID":"e9fd1729-6f5b-4fb4-9240-e513cabed3b6","Type":"ContainerStarted","Data":"b535c3fcb7121266f2885e6337a8f74a43ef6588f0d6992788c69b782ecf9351"}
Sep 18 18:20:51.783392 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:20:51.783250    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:21:00.834612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.834579    2550 generic.go:356] "Generic (PLEG): container finished" podID="c4e29416-e22e-483f-9647-233a72bee579" containerID="1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25" exitCode=0
Sep 18 18:21:00.835102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.834653    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerDied","Data":"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"}
Sep 18 18:21:00.835102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.835024    2550 scope.go:117] "RemoveContainer" containerID="1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"
Sep 18 18:21:00.836231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.836205    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:00.836305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.836270    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:00.836305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.836303    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:00.853963 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:00.853225    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" podStartSLOduration=10.85320854 podStartE2EDuration="10.85320854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:20:51.799472825 +0000 UTC m=+1223.486102299" watchObservedRunningTime="2026-09-18 18:21:00.85320854 +0000 UTC m=+1232.539837993"
Sep 18 18:21:01.848765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:01.848726    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerStarted","Data":"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"}
Sep 18 18:21:01.906818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:01.906776    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:21:01.907005 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:01.906926    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:01.907089 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:01.906997    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:02.791427 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:02.791399    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:21:02.906792 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:02.906758    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:02.907194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:02.906810    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:03.907030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:03.906991    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:03.907470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:03.907051    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:04.907373 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:04.907329    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:04.907766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:04.907398    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:05.907161 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:05.907119    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:05.907456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:05.907427    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:06.907465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:06.907424    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:06.907863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:06.907497    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:07.621234 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:07.621182    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:21:07.907609 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:07.907518    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 18 18:21:07.907609 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:07.907590    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 18 18:21:08.911590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:08.911559    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:21:08.912542 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:08.912519    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:21:14.939415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.939382    2550 generic.go:356] "Generic (PLEG): container finished" podID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerID="ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450" exitCode=0
Sep 18 18:21:14.939823 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.939452    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerDied","Data":"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"}
Sep 18 18:21:14.939823 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.939817    2550 scope.go:117] "RemoveContainer" containerID="ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"
Sep 18 18:21:14.941020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.940995    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:14.941108 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.941061    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:14.941108 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:14.941094    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:21:15.950583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:15.950542    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerStarted","Data":"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"}
Sep 18 18:21:16.400711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:16.400673    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:21:16.401698 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:16.401672    2550 patch_prober.go:28] interesting pod/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 18:21:16.401833 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:16.401721    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 18:21:17.401191 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:17.401150    2550 patch_prober.go:28] interesting pod/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 18:21:17.401590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:17.401224    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 18:21:17.759692 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:17.759658    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:21:18.401195 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:18.401158    2550 patch_prober.go:28] interesting pod/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 18:21:18.401585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:18.401221    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 18:21:19.400552 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:19.400516    2550 patch_prober.go:28] interesting pod/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 18:21:19.400802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:19.400581    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 18:21:20.400795 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:20.400760    2550 patch_prober.go:28] interesting pod/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Sep 18 18:21:20.401205 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:20.400825    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Sep 18 18:21:21.404708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:21.404667    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:21:21.405569 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:21.405551    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:21:25.886772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:25.886738    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:21:30.828788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:30.828752    2550 scope.go:117] "RemoveContainer" containerID="b78873c6ef292cda1864f18a6f5d9c9586816ad18761f19c59ad920e5417199a"
Sep 18 18:21:32.423691 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.423659    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"]
Sep 18 18:21:32.428358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.428341    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:32.430705 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.430688    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:21:32.430794 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.430734    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:21:32.434830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.434806    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"]
Sep 18 18:21:32.522418 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.522391    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp\" (UID: \"25f90ca8-01d0-4e2a-a728-43cbec298285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:32.623786 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.623759    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp\" (UID: \"25f90ca8-01d0-4e2a-a728-43cbec298285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:32.624348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.624328    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp\" (UID: \"25f90ca8-01d0-4e2a-a728-43cbec298285\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:32.740446 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:32.740390    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:33.013041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:33.013016    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"]
Sep 18 18:21:33.014446 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:21:33.014421    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25f90ca8_01d0_4e2a_a728_43cbec298285.slice/crio-687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92 WatchSource:0}: Error finding container 687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92: Status 404 returned error can't find the container with id 687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92
Sep 18 18:21:33.051229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:33.051204    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp" event={"ID":"25f90ca8-01d0-4e2a-a728-43cbec298285","Type":"ContainerStarted","Data":"687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92"}
Sep 18 18:21:34.058519 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:34.058447    2550 generic.go:356] "Generic (PLEG): container finished" podID="25f90ca8-01d0-4e2a-a728-43cbec298285" containerID="d5c60b95d86392647f732f7bbfb45fe1123c3f0a2385ab87f53d5f48c90e8b59" exitCode=0
Sep 18 18:21:34.058851 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:34.058524    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp" event={"ID":"25f90ca8-01d0-4e2a-a728-43cbec298285","Type":"ContainerDied","Data":"d5c60b95d86392647f732f7bbfb45fe1123c3f0a2385ab87f53d5f48c90e8b59"}
Sep 18 18:21:35.203382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:35.203358    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:35.351138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:35.351062    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca\") pod \"25f90ca8-01d0-4e2a-a728-43cbec298285\" (UID: \"25f90ca8-01d0-4e2a-a728-43cbec298285\") "
Sep 18 18:21:35.351425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:35.351405    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "25f90ca8-01d0-4e2a-a728-43cbec298285" (UID: "25f90ca8-01d0-4e2a-a728-43cbec298285"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:21:35.452337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:35.452305    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/25f90ca8-01d0-4e2a-a728-43cbec298285-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:36.071481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:36.071448    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp" event={"ID":"25f90ca8-01d0-4e2a-a728-43cbec298285","Type":"ContainerDied","Data":"687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92"}
Sep 18 18:21:36.071481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:36.071476    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"
Sep 18 18:21:36.071689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:36.071505    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="687694a2373bc6a7936bc606d9f1f6b03a4dc84f7c7e2c9643a629dee809db92"
Sep 18 18:21:37.293251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:37.293217    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:21:37.302241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:37.302215    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:21:37.319148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:37.319119    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:21:47.997611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:47.997573    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:21:47.998152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:47.997877    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" podUID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" containerName="payload-processing" containerID="cri-o://25636c6ab7aa7f3b716677eb1f0c5f5c831c07eda01c3a2ed6d1c77b0e92b4d0" gracePeriod=30
Sep 18 18:21:48.002358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.002331    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:21:48.002735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.002703    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" podUID="f446ffe6-f0d2-4708-b318-ceea6debf39c" containerName="payload-pre-processing" containerID="cri-o://44492a964ae22372fe879027489982ab4cdcbe284b0a6c40eb70e1aac98fa434" gracePeriod=30
Sep 18 18:21:48.138638 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.138607    2550 generic.go:356] "Generic (PLEG): container finished" podID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" containerID="25636c6ab7aa7f3b716677eb1f0c5f5c831c07eda01c3a2ed6d1c77b0e92b4d0" exitCode=0
Sep 18 18:21:48.138802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.138688    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" event={"ID":"87ed5217-fc2c-4725-a1da-6b6a0b9136a0","Type":"ContainerDied","Data":"25636c6ab7aa7f3b716677eb1f0c5f5c831c07eda01c3a2ed6d1c77b0e92b4d0"}
Sep 18 18:21:48.140649 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.140624    2550 generic.go:356] "Generic (PLEG): container finished" podID="f446ffe6-f0d2-4708-b318-ceea6debf39c" containerID="44492a964ae22372fe879027489982ab4cdcbe284b0a6c40eb70e1aac98fa434" exitCode=0
Sep 18 18:21:48.140768 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.140713    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" event={"ID":"f446ffe6-f0d2-4708-b318-ceea6debf39c","Type":"ContainerDied","Data":"44492a964ae22372fe879027489982ab4cdcbe284b0a6c40eb70e1aac98fa434"}
Sep 18 18:21:48.256701 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.256648    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:21:48.273698 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.273676    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:21:48.323075 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.323047    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"]
Sep 18 18:21:48.325809 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.325787    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-039dbc-wzpdp"]
Sep 18 18:21:48.365719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.365683    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume\") pod \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") "
Sep 18 18:21:48.365719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.365721    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj9px\" (UniqueName: \"kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px\") pod \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\" (UID: \"87ed5217-fc2c-4725-a1da-6b6a0b9136a0\") "
Sep 18 18:21:48.365919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.365766    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume\") pod \"f446ffe6-f0d2-4708-b318-ceea6debf39c\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") "
Sep 18 18:21:48.365919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.365825    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhksw\" (UniqueName: \"kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw\") pod \"f446ffe6-f0d2-4708-b318-ceea6debf39c\" (UID: \"f446ffe6-f0d2-4708-b318-ceea6debf39c\") "
Sep 18 18:21:48.366154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.366122    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87ed5217-fc2c-4725-a1da-6b6a0b9136a0" (UID: "87ed5217-fc2c-4725-a1da-6b6a0b9136a0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:21:48.366294 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.366202    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f446ffe6-f0d2-4708-b318-ceea6debf39c" (UID: "f446ffe6-f0d2-4708-b318-ceea6debf39c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:21:48.366294 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.366283    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:48.368042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.368015    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw" (OuterVolumeSpecName: "kube-api-access-vhksw") pod "f446ffe6-f0d2-4708-b318-ceea6debf39c" (UID: "f446ffe6-f0d2-4708-b318-ceea6debf39c"). InnerVolumeSpecName "kube-api-access-vhksw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:21:48.368148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.368049    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px" (OuterVolumeSpecName: "kube-api-access-nj9px") pod "87ed5217-fc2c-4725-a1da-6b6a0b9136a0" (UID: "87ed5217-fc2c-4725-a1da-6b6a0b9136a0"). InnerVolumeSpecName "kube-api-access-nj9px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:21:48.467621 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.467580    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj9px\" (UniqueName: \"kubernetes.io/projected/87ed5217-fc2c-4725-a1da-6b6a0b9136a0-kube-api-access-nj9px\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:48.467621 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.467622    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f446ffe6-f0d2-4708-b318-ceea6debf39c-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:48.467815 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.467638    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhksw\" (UniqueName: \"kubernetes.io/projected/f446ffe6-f0d2-4708-b318-ceea6debf39c-kube-api-access-vhksw\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:48.706962 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:48.706915    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25f90ca8-01d0-4e2a-a728-43cbec298285" path="/var/lib/kubelet/pods/25f90ca8-01d0-4e2a-a728-43cbec298285/volumes"
Sep 18 18:21:49.147592 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.147560    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"
Sep 18 18:21:49.148047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.147589    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm" event={"ID":"87ed5217-fc2c-4725-a1da-6b6a0b9136a0","Type":"ContainerDied","Data":"d5402d16e50a2651106cc900a13ad5dae5bc21614e91053da29b53dc5ba82ee0"}
Sep 18 18:21:49.148047 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.147663    2550 scope.go:117] "RemoveContainer" containerID="25636c6ab7aa7f3b716677eb1f0c5f5c831c07eda01c3a2ed6d1c77b0e92b4d0"
Sep 18 18:21:49.149148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.149124    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"
Sep 18 18:21:49.149148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.149135    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z" event={"ID":"f446ffe6-f0d2-4708-b318-ceea6debf39c","Type":"ContainerDied","Data":"d8c666a3cfd6465d6d90e3717130cd39a93f9f0973573d9d762e72d1b88f8046"}
Sep 18 18:21:49.158947 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.158919    2550 scope.go:117] "RemoveContainer" containerID="44492a964ae22372fe879027489982ab4cdcbe284b0a6c40eb70e1aac98fa434"
Sep 18 18:21:49.166162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.166141    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:21:49.169183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.169164    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-039dbc-c8b479995-b22vm"]
Sep 18 18:21:49.179764 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.179742    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:21:49.183358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:49.183338    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-039dbc-77687f74bb-67x8z"]
Sep 18 18:21:50.707146 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:50.707115    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" path="/var/lib/kubelet/pods/87ed5217-fc2c-4725-a1da-6b6a0b9136a0/volumes"
Sep 18 18:21:50.707574 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:50.707482    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f446ffe6-f0d2-4708-b318-ceea6debf39c" path="/var/lib/kubelet/pods/f446ffe6-f0d2-4708-b318-ceea6debf39c/volumes"
Sep 18 18:21:51.580523 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.580492    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:21:51.580927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.580909    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25f90ca8-01d0-4e2a-a728-43cbec298285" containerName="revoke-keys"
Sep 18 18:21:51.581010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.580932    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="25f90ca8-01d0-4e2a-a728-43cbec298285" containerName="revoke-keys"
Sep 18 18:21:51.581010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.580979    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f446ffe6-f0d2-4708-b318-ceea6debf39c" containerName="payload-pre-processing"
Sep 18 18:21:51.581010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.580995    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f446ffe6-f0d2-4708-b318-ceea6debf39c" containerName="payload-pre-processing"
Sep 18 18:21:51.581110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.581014    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" containerName="payload-processing"
Sep 18 18:21:51.581110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.581021    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" containerName="payload-processing"
Sep 18 18:21:51.581110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.581095    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ed5217-fc2c-4725-a1da-6b6a0b9136a0" containerName="payload-processing"
Sep 18 18:21:51.581110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.581101    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="25f90ca8-01d0-4e2a-a728-43cbec298285" containerName="revoke-keys"
Sep 18 18:21:51.581243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.581111    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f446ffe6-f0d2-4708-b318-ceea6debf39c" containerName="payload-pre-processing"
Sep 18 18:21:51.585222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.585207    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.587387 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.587365    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-011541\"/\"kube-root-ca.crt\""
Sep 18 18:21:51.588639 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.588528    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-011541\"/\"default-dockercfg-zdxr6\""
Sep 18 18:21:51.588639 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.588553    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-011541\"/\"openshift-service-ca.crt\""
Sep 18 18:21:51.588867 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.588847    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-011541\"/\"auth-test-model-011541-kserve-self-signed-certs\""
Sep 18 18:21:51.591915 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.591890    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:21:51.696216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696184    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.696369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696222    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.696369 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696325    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.696494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696369    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.696494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696398    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rxht\" (UniqueName: \"kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.696494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.696452    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797267    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797314    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797339    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rxht\" (UniqueName: \"kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797399    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797475    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.797734 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797511    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.798038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797741    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.798038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797965    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.798038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.797976    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.800285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.800256    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.800486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.800469    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.806158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.806135    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rxht\" (UniqueName: \"kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht\") pod \"auth-test-model-011541-kserve-55989c4ff5-kr5jz\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") " pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:51.895853 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:51.895802    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:21:52.173433 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.173355    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:21:52.178578 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:21:52.178549    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda87885eb_8f12_4795_b251_e45129ba1ab8.slice/crio-95fb1838114b922eaded7836c6e0f5f620f80cad3dd66832dbcc10181a2c17b5 WatchSource:0}: Error finding container 95fb1838114b922eaded7836c6e0f5f620f80cad3dd66832dbcc10181a2c17b5: Status 404 returned error can't find the container with id 95fb1838114b922eaded7836c6e0f5f620f80cad3dd66832dbcc10181a2c17b5
Sep 18 18:21:52.313895 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.313842    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" podUID="e67a5606-81a2-4e18-9a79-412047699491" containerName="main" containerID="cri-o://34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1" gracePeriod=45
Sep 18 18:21:52.325397 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.325301    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" podUID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" containerName="main" containerID="cri-o://4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6" gracePeriod=45
Sep 18 18:21:52.339103 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.339073    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" podUID="c4794c02-448d-44ca-8d43-a308cb3be0ea" containerName="main" containerID="cri-o://1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee" gracePeriod=45
Sep 18 18:21:52.614920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.614896    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:21:52.619609 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.619581    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:21:52.625445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.625423    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:21:52.704880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704803    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.704880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704842    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.704880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704863    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704890    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704911    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704966    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.704995    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705022    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705072    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705111    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705113    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705242    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgqt6\" (UniqueName: \"kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705270    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705283    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705298    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home" (OuterVolumeSpecName: "home") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705330    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705363    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdjcl\" (UniqueName: \"kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705398    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705402    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache" (OuterVolumeSpecName: "model-cache") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705423    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25fgc\" (UniqueName: \"kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc\") pod \"e67a5606-81a2-4e18-9a79-412047699491\" (UID: \"e67a5606-81a2-4e18-9a79-412047699491\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705453    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm\") pod \"c4794c02-448d-44ca-8d43-a308cb3be0ea\" (UID: \"c4794c02-448d-44ca-8d43-a308cb3be0ea\") "
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705491    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705503    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.705612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705524    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs\") pod \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\" (UID: \"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1\") "
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705706    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home" (OuterVolumeSpecName: "home") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705924    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.705999    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706012    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706024    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706036    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706048    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706059    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.706293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706272    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.706852 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.706827    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home" (OuterVolumeSpecName: "home") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.707716 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.707677    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:21:52.708281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.708251    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm" (OuterVolumeSpecName: "dshm") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.708686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.708659    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm" (OuterVolumeSpecName: "dshm") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.708686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.708674    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:21:52.708993 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.708972    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl" (OuterVolumeSpecName: "kube-api-access-wdjcl") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "kube-api-access-wdjcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:21:52.709098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.709077    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6" (OuterVolumeSpecName: "kube-api-access-kgqt6") pod "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" (UID: "0c5dad5e-c9ae-49df-89da-9d9392c8c1f1"). InnerVolumeSpecName "kube-api-access-kgqt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:21:52.709197 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.709181    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm" (OuterVolumeSpecName: "dshm") pod "c4794c02-448d-44ca-8d43-a308cb3be0ea" (UID: "c4794c02-448d-44ca-8d43-a308cb3be0ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:21:52.709331 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.709316    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:21:52.709836 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.709820    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc" (OuterVolumeSpecName: "kube-api-access-25fgc") pod "e67a5606-81a2-4e18-9a79-412047699491" (UID: "e67a5606-81a2-4e18-9a79-412047699491"). InnerVolumeSpecName "kube-api-access-25fgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:21:52.806761 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806738    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e67a5606-81a2-4e18-9a79-412047699491-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.806761 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806761    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdjcl\" (UniqueName: \"kubernetes.io/projected/c4794c02-448d-44ca-8d43-a308cb3be0ea-kube-api-access-wdjcl\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806772    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806781    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25fgc\" (UniqueName: \"kubernetes.io/projected/e67a5606-81a2-4e18-9a79-412047699491-kube-api-access-25fgc\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806789    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4794c02-448d-44ca-8d43-a308cb3be0ea-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806797    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806806    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806813    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4794c02-448d-44ca-8d43-a308cb3be0ea-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806820    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e67a5606-81a2-4e18-9a79-412047699491-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806827    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgqt6\" (UniqueName: \"kubernetes.io/projected/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-kube-api-access-kgqt6\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:52.807125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:52.806834    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:21:53.177342 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.177315    2550 generic.go:356] "Generic (PLEG): container finished" podID="e67a5606-81a2-4e18-9a79-412047699491" containerID="34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1" exitCode=0
Sep 18 18:21:53.177498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.177357    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"
Sep 18 18:21:53.177498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.177395    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" event={"ID":"e67a5606-81a2-4e18-9a79-412047699491","Type":"ContainerDied","Data":"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"}
Sep 18 18:21:53.177498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.177434    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf" event={"ID":"e67a5606-81a2-4e18-9a79-412047699491","Type":"ContainerDied","Data":"29db2111526d1ecd99a4f05f798c97d4c4f95c8ea4d1aee97a6d51af7f8161ce"}
Sep 18 18:21:53.177498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.177450    2550 scope.go:117] "RemoveContainer" containerID="34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"
Sep 18 18:21:53.178913 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.178889    2550 generic.go:356] "Generic (PLEG): container finished" podID="c4794c02-448d-44ca-8d43-a308cb3be0ea" containerID="1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee" exitCode=0
Sep 18 18:21:53.179081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.178929    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"
Sep 18 18:21:53.179081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.178976    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" event={"ID":"c4794c02-448d-44ca-8d43-a308cb3be0ea","Type":"ContainerDied","Data":"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"}
Sep 18 18:21:53.179081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.179023    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq" event={"ID":"c4794c02-448d-44ca-8d43-a308cb3be0ea","Type":"ContainerDied","Data":"df61e6e1dc5770b4271f0dd00ac2ad39da2fe70c6c778684881a997037f7920c"}
Sep 18 18:21:53.180994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.180969    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" event={"ID":"a87885eb-8f12-4795-b251-e45129ba1ab8","Type":"ContainerStarted","Data":"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"}
Sep 18 18:21:53.181122 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.181015    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" event={"ID":"a87885eb-8f12-4795-b251-e45129ba1ab8","Type":"ContainerStarted","Data":"95fb1838114b922eaded7836c6e0f5f620f80cad3dd66832dbcc10181a2c17b5"}
Sep 18 18:21:53.182477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.182449    2550 generic.go:356] "Generic (PLEG): container finished" podID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" containerID="4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6" exitCode=0
Sep 18 18:21:53.182591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.182479    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"
Sep 18 18:21:53.182591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.182520    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" event={"ID":"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1","Type":"ContainerDied","Data":"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"}
Sep 18 18:21:53.182591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.182549    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc" event={"ID":"0c5dad5e-c9ae-49df-89da-9d9392c8c1f1","Type":"ContainerDied","Data":"ea45450fbac5b28c6a99ce5aa7f36cee96f17b84f78950310c2db59be5d01264"}
Sep 18 18:21:53.188239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.188219    2550 scope.go:117] "RemoveContainer" containerID="34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"
Sep 18 18:21:53.188487 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:21:53.188468    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1\": container with ID starting with 34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1 not found: ID does not exist" containerID="34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"
Sep 18 18:21:53.188532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.188494    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1"} err="failed to get container status \"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1\": rpc error: code = NotFound desc = could not find container \"34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1\": container with ID starting with 34d8d4bf5681ea9aaab46ac6330ef18d7a9b7e5a8dd2a955c05c6eeba3243cd1 not found: ID does not exist"
Sep 18 18:21:53.188532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.188512    2550 scope.go:117] "RemoveContainer" containerID="1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"
Sep 18 18:21:53.197303 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.197282    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:21:53.197671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.197653    2550 scope.go:117] "RemoveContainer" containerID="1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"
Sep 18 18:21:53.197975 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:21:53.197915    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee\": container with ID starting with 1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee not found: ID does not exist" containerID="1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"
Sep 18 18:21:53.197975 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.197958    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee"} err="failed to get container status \"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee\": rpc error: code = NotFound desc = could not find container \"1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee\": container with ID starting with 1975d7e520ac8ba9b1e8481efa0e37abbc997381b54500b16d5b627b0b85eeee not found: ID does not exist"
Sep 18 18:21:53.197975 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.197975    2550 scope.go:117] "RemoveContainer" containerID="4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"
Sep 18 18:21:53.201219 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.201198    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-2-w3-039dbc-kserve-649f465ccb-c9hsf"]
Sep 18 18:21:53.205703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.205688    2550 scope.go:117] "RemoveContainer" containerID="4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"
Sep 18 18:21:53.205974 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:21:53.205953    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6\": container with ID starting with 4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6 not found: ID does not exist" containerID="4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"
Sep 18 18:21:53.206029 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.205982    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6"} err="failed to get container status \"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6\": rpc error: code = NotFound desc = could not find container \"4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6\": container with ID starting with 4487908bf501a47cb449513814f57306c495ac2134a4a79e065191ddbff572b6 not found: ID does not exist"
Sep 18 18:21:53.214704 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.214666    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" podStartSLOduration=2.214655017 podStartE2EDuration="2.214655017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:21:53.213316496 +0000 UTC m=+1284.899945972" watchObservedRunningTime="2026-09-18 18:21:53.214655017 +0000 UTC m=+1284.901284469"
Sep 18 18:21:53.226959 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.226918    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:21:53.228761 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.228740    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/distinct-w3-039dbc-kserve-5f68c8df95-4qrbc"]
Sep 18 18:21:53.236969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.236946    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:21:53.241396 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:53.241378    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-039dbc/simulator-w3-039dbc-kserve-6c568ff5d9-qfxdq"]
Sep 18 18:21:54.707098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:54.707069    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" path="/var/lib/kubelet/pods/0c5dad5e-c9ae-49df-89da-9d9392c8c1f1/volumes"
Sep 18 18:21:54.707527 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:54.707510    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4794c02-448d-44ca-8d43-a308cb3be0ea" path="/var/lib/kubelet/pods/c4794c02-448d-44ca-8d43-a308cb3be0ea/volumes"
Sep 18 18:21:54.707898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:21:54.707886    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67a5606-81a2-4e18-9a79-412047699491" path="/var/lib/kubelet/pods/e67a5606-81a2-4e18-9a79-412047699491/volumes"
Sep 18 18:22:01.896149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:01.896118    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:22:01.896514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:01.896171    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:22:01.908220 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:01.908193    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:22:02.247867 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:02.247791    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:22:06.589075 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:06.589026    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:22:23.713076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713045    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"]
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713434    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4794c02-448d-44ca-8d43-a308cb3be0ea" containerName="main"
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713444    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4794c02-448d-44ca-8d43-a308cb3be0ea" containerName="main"
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713466    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" containerName="main"
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713472    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" containerName="main"
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713483    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67a5606-81a2-4e18-9a79-412047699491" containerName="main"
Sep 18 18:22:23.713494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713488    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67a5606-81a2-4e18-9a79-412047699491" containerName="main"
Sep 18 18:22:23.713697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713566    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4794c02-448d-44ca-8d43-a308cb3be0ea" containerName="main"
Sep 18 18:22:23.713697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713575    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5dad5e-c9ae-49df-89da-9d9392c8c1f1" containerName="main"
Sep 18 18:22:23.713697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.713583    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67a5606-81a2-4e18-9a79-412047699491" containerName="main"
Sep 18 18:22:23.716469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.716453    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:23.718963 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.718929    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:22:23.719075 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.718965    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:22:23.724405 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.724382    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"]
Sep 18 18:22:23.789741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.789706    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh\" (UID: \"5819b9f1-6f94-495e-bfdf-2e2994050d3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:23.891063 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.891025    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh\" (UID: \"5819b9f1-6f94-495e-bfdf-2e2994050d3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:23.891654 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:23.891635    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh\" (UID: \"5819b9f1-6f94-495e-bfdf-2e2994050d3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:24.028996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:24.028969    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:24.299726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:24.299528    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"]
Sep 18 18:22:24.302499 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:22:24.302463    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5819b9f1_6f94_495e_bfdf_2e2994050d3e.slice/crio-87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169 WatchSource:0}: Error finding container 87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169: Status 404 returned error can't find the container with id 87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169
Sep 18 18:22:24.353655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:24.353627    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh" event={"ID":"5819b9f1-6f94-495e-bfdf-2e2994050d3e","Type":"ContainerStarted","Data":"87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169"}
Sep 18 18:22:25.361316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:25.361232    2550 generic.go:356] "Generic (PLEG): container finished" podID="5819b9f1-6f94-495e-bfdf-2e2994050d3e" containerID="b2e737773d6fba3bf245de359fece88920ff891c1996943d3b8e2d2d5e045acd" exitCode=0
Sep 18 18:22:25.361316 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:25.361282    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh" event={"ID":"5819b9f1-6f94-495e-bfdf-2e2994050d3e","Type":"ContainerDied","Data":"b2e737773d6fba3bf245de359fece88920ff891c1996943d3b8e2d2d5e045acd"}
Sep 18 18:22:26.501740 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:26.501717    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:26.514416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:26.514392    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca\") pod \"5819b9f1-6f94-495e-bfdf-2e2994050d3e\" (UID: \"5819b9f1-6f94-495e-bfdf-2e2994050d3e\") "
Sep 18 18:22:26.514697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:26.514677    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5819b9f1-6f94-495e-bfdf-2e2994050d3e" (UID: "5819b9f1-6f94-495e-bfdf-2e2994050d3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:22:26.514808 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:26.514796    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5819b9f1-6f94-495e-bfdf-2e2994050d3e-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:22:27.373280 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:27.373253    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"
Sep 18 18:22:27.373463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:27.373253    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh" event={"ID":"5819b9f1-6f94-495e-bfdf-2e2994050d3e","Type":"ContainerDied","Data":"87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169"}
Sep 18 18:22:27.373463 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:27.373403    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87076151f5ed485a05f6385a3c1d98db58e32953f6b6f47b969ef2935c767169"
Sep 18 18:22:30.087416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:30.087380    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:22:30.851584 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:30.851553    2550 scope.go:117] "RemoveContainer" containerID="35763ef421c524404a2c54ee0d24bb7d383c17873fcd7dd99284d0b397b14d36"
Sep 18 18:22:31.489166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.489133    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:22:31.489549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.489535    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5819b9f1-6f94-495e-bfdf-2e2994050d3e" containerName="revoke-keys"
Sep 18 18:22:31.489549 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.489550    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5819b9f1-6f94-495e-bfdf-2e2994050d3e" containerName="revoke-keys"
Sep 18 18:22:31.489630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.489611    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5819b9f1-6f94-495e-bfdf-2e2994050d3e" containerName="revoke-keys"
Sep 18 18:22:31.494386 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.494362    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.497675 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.497649    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"default-dockercfg-r7zmk\""
Sep 18 18:22:31.497875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.497660    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"kube-root-ca.crt\""
Sep 18 18:22:31.497875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.497659    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"openshift-service-ca.crt\""
Sep 18 18:22:31.497875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.497722    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"auth-test-model-99602c-kserve-self-signed-certs\""
Sep 18 18:22:31.503694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.503670    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:22:31.562068 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562039    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.562207 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562086    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.562207 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562125    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.562324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562214    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmcp\" (UniqueName: \"kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.562324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562264    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.562324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.562286    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663262    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmcp\" (UniqueName: \"kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663307    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663330    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663356    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663396    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663430    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663756    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663768    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.663869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.663849    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.665629 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.665610    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.665744 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.665720    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.670425 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.670404    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmcp\" (UniqueName: \"kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp\") pod \"auth-test-model-99602c-kserve-6c99db86c5-n5m9j\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") " pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:31.805107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:31.805079    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:32.078617 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:32.078594    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:22:32.080683 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:22:32.080658    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f334b50_f913_42f1_a32c_ff5e12335b45.slice/crio-a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46 WatchSource:0}: Error finding container a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46: Status 404 returned error can't find the container with id a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46
Sep 18 18:22:32.404378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:32.404293    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" event={"ID":"0f334b50-f913-42f1-a32c-ff5e12335b45","Type":"ContainerStarted","Data":"bcbd06c2c10054e5cdda2be1e44a81999af2056622fde0a70be0db272d13f0a7"}
Sep 18 18:22:32.404378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:32.404345    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" event={"ID":"0f334b50-f913-42f1-a32c-ff5e12335b45","Type":"ContainerStarted","Data":"a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46"}
Sep 18 18:22:32.421975 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:32.421917    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" podStartSLOduration=1.4219018829999999 podStartE2EDuration="1.421901883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:22:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:22:32.420101031 +0000 UTC m=+1324.106730503" watchObservedRunningTime="2026-09-18 18:22:32.421901883 +0000 UTC m=+1324.108531334"
Sep 18 18:22:39.333690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.333654    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:22:39.334572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.334534    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" podUID="f6107d2c-5a40-4814-aa37-cfb9277b831b" containerName="payload-pre-processing" containerID="cri-o://c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0" gracePeriod=30
Sep 18 18:22:39.515406 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.515373    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"]
Sep 18 18:22:39.517395 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.517373    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6fa1a2a2-4nfsh"]
Sep 18 18:22:39.603665 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.603610    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:22:39.734782 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.734755    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume\") pod \"f6107d2c-5a40-4814-aa37-cfb9277b831b\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") "
Sep 18 18:22:39.734926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.734798    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7hjz\" (UniqueName: \"kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz\") pod \"f6107d2c-5a40-4814-aa37-cfb9277b831b\" (UID: \"f6107d2c-5a40-4814-aa37-cfb9277b831b\") "
Sep 18 18:22:39.735132 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.735111    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6107d2c-5a40-4814-aa37-cfb9277b831b" (UID: "f6107d2c-5a40-4814-aa37-cfb9277b831b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:22:39.737051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.737029    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz" (OuterVolumeSpecName: "kube-api-access-f7hjz") pod "f6107d2c-5a40-4814-aa37-cfb9277b831b" (UID: "f6107d2c-5a40-4814-aa37-cfb9277b831b"). InnerVolumeSpecName "kube-api-access-f7hjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:22:39.836189 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.836167    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6107d2c-5a40-4814-aa37-cfb9277b831b-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:22:39.836189 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:39.836188    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7hjz\" (UniqueName: \"kubernetes.io/projected/f6107d2c-5a40-4814-aa37-cfb9277b831b-kube-api-access-f7hjz\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:22:40.451645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.451605    2550 generic.go:356] "Generic (PLEG): container finished" podID="f6107d2c-5a40-4814-aa37-cfb9277b831b" containerID="c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0" exitCode=0
Sep 18 18:22:40.451645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.451644    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"
Sep 18 18:22:40.452190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.451695    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" event={"ID":"f6107d2c-5a40-4814-aa37-cfb9277b831b","Type":"ContainerDied","Data":"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"}
Sep 18 18:22:40.452190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.451743    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt" event={"ID":"f6107d2c-5a40-4814-aa37-cfb9277b831b","Type":"ContainerDied","Data":"b05d3bee0bc24e2b30888c304b0edceda089cd6cdd479cf4a65f48441a7142e1"}
Sep 18 18:22:40.452190 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.451765    2550 scope.go:117] "RemoveContainer" containerID="c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"
Sep 18 18:22:40.461423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.461406    2550 scope.go:117] "RemoveContainer" containerID="c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"
Sep 18 18:22:40.461695 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:22:40.461675    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0\": container with ID starting with c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0 not found: ID does not exist" containerID="c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"
Sep 18 18:22:40.461745 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.461703    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0"} err="failed to get container status \"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0\": rpc error: code = NotFound desc = could not find container \"c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0\": container with ID starting with c11f64c9c4905f80e1327de647753d71a82fb4cf3faa3cc27c7e934eecb2c8c0 not found: ID does not exist"
Sep 18 18:22:40.472193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.472168    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:22:40.476038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.476017    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6fa1a2a2-97598df86-dhdzt"]
Sep 18 18:22:40.706979 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.706899    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5819b9f1-6f94-495e-bfdf-2e2994050d3e" path="/var/lib/kubelet/pods/5819b9f1-6f94-495e-bfdf-2e2994050d3e/volumes"
Sep 18 18:22:40.707271 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:40.707257    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6107d2c-5a40-4814-aa37-cfb9277b831b" path="/var/lib/kubelet/pods/f6107d2c-5a40-4814-aa37-cfb9277b831b/volumes"
Sep 18 18:22:41.805908 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:41.805875    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:41.805908 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:41.805906    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:41.818462 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:41.818434    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:22:42.473774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:22:42.473742    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:23:02.192538 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.192503    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:23:02.192950 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.192900    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6107d2c-5a40-4814-aa37-cfb9277b831b" containerName="payload-pre-processing"
Sep 18 18:23:02.192950 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.192911    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6107d2c-5a40-4814-aa37-cfb9277b831b" containerName="payload-pre-processing"
Sep 18 18:23:02.193067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.193048    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6107d2c-5a40-4814-aa37-cfb9277b831b" containerName="payload-pre-processing"
Sep 18 18:23:02.196254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.196238    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.198723 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.198696    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-37c153\"/\"embedding-w0-37c153-kserve-self-signed-certs\""
Sep 18 18:23:02.202925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.202897    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:23:02.245858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.245834    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.245993 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.245890    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.246067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.246005    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w9cg\" (UniqueName: \"kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.246067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.246036    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.246067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.246052    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.246225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.246073    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347277    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347324    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w9cg\" (UniqueName: \"kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347343    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347361    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347379    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347402    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347745 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347724    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347749    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.347867 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.347802    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.349480 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.349462    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.349742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.349726    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.355878 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.355853    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w9cg\" (UniqueName: \"kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg\") pod \"embedding-w0-37c153-kserve-f89c586-v2wzt\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") " pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.507770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.507750    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:02.684241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.684212    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:23:02.786445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:02.786424    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:23:02.788151 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:23:02.788124    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ef6eef1_4076_45d5_823e_a3465ba5daa9.slice/crio-52b1563fb500eacd02201c7002db8d205c9caf5585c0fb3f3dc928b5e7453777 WatchSource:0}: Error finding container 52b1563fb500eacd02201c7002db8d205c9caf5585c0fb3f3dc928b5e7453777: Status 404 returned error can't find the container with id 52b1563fb500eacd02201c7002db8d205c9caf5585c0fb3f3dc928b5e7453777
Sep 18 18:23:03.574368 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:03.574332    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" event={"ID":"9ef6eef1-4076-45d5-823e-a3465ba5daa9","Type":"ContainerStarted","Data":"231a5768d5d23eda77d108da57164fad377fe1865d442c6725e2d935d21bfafb"}
Sep 18 18:23:03.574753 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:03.574383    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" event={"ID":"9ef6eef1-4076-45d5-823e-a3465ba5daa9","Type":"ContainerStarted","Data":"52b1563fb500eacd02201c7002db8d205c9caf5585c0fb3f3dc928b5e7453777"}
Sep 18 18:23:03.591873 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:03.591824    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" podStartSLOduration=1.5917899119999999 podStartE2EDuration="1.591789912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:23:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:23:03.589632991 +0000 UTC m=+1355.276262446" watchObservedRunningTime="2026-09-18 18:23:03.591789912 +0000 UTC m=+1355.278419363"
Sep 18 18:23:12.508037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:12.507933    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:12.508496 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:12.508053    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:12.520508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:12.520483    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:12.635388 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:12.635364    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:23:25.090273 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:25.090236    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:23:35.999956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:35.999897    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:23:36.000443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:36.000159    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" containerID="cri-o://a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3" gracePeriod=30
Sep 18 18:23:38.914589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:38.914551    2550 patch_prober.go:28] interesting pod/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:23:38.914985 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:38.914609    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:23:41.250799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.250766    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:23:41.384364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384306    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384343    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384374    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msmlz\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384399    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384414    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384437    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384465    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384490    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.384841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384602    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:23:41.384841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384755    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:23:41.384841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384770    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:23:41.385009 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384819    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket\") pod \"c4e29416-e22e-483f-9647-233a72bee579\" (UID: \"c4e29416-e22e-483f-9647-233a72bee579\") "
Sep 18 18:23:41.385009 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.384999    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:23:41.385139 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385115    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:23:41.385306 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385276    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.385306 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385299    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.385485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385312    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.385485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385325    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4e29416-e22e-483f-9647-233a72bee579-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.385485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.385336    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.386789 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.386756    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:23:41.386893 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.386821    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz" (OuterVolumeSpecName: "kube-api-access-msmlz") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "kube-api-access-msmlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:23:41.387066 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.387047    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:23:41.387135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.387073    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4e29416-e22e-483f-9647-233a72bee579" (UID: "c4e29416-e22e-483f-9647-233a72bee579"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:23:41.485678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.485654    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4e29416-e22e-483f-9647-233a72bee579-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.485678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.485676    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msmlz\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-kube-api-access-msmlz\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.485795 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.485686    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4e29416-e22e-483f-9647-233a72bee579-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.485795 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.485695    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4e29416-e22e-483f-9647-233a72bee579-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:23:41.783443 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.783408    2550 generic.go:356] "Generic (PLEG): container finished" podID="c4e29416-e22e-483f-9647-233a72bee579" containerID="a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3" exitCode=0
Sep 18 18:23:41.783588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.783478    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerDied","Data":"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"}
Sep 18 18:23:41.783588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.783546    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"
Sep 18 18:23:41.783588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.783561    2550 scope.go:117] "RemoveContainer" containerID="a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"
Sep 18 18:23:41.783751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.783546    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st" event={"ID":"c4e29416-e22e-483f-9647-233a72bee579","Type":"ContainerDied","Data":"7c88e0fca27ac1664a7f91cd0839a80f80f0b5f3c6105584a98b363ca5ebc8ec"}
Sep 18 18:23:41.794062 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.794043    2550 scope.go:117] "RemoveContainer" containerID="1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"
Sep 18 18:23:41.804783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.804766    2550 scope.go:117] "RemoveContainer" containerID="a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"
Sep 18 18:23:41.805147 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:23:41.805128    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3\": container with ID starting with a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3 not found: ID does not exist" containerID="a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"
Sep 18 18:23:41.805237 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.805158    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3"} err="failed to get container status \"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3\": rpc error: code = NotFound desc = could not find container \"a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3\": container with ID starting with a9d824f0843d2fb0ab20cf31e961dfa6d7537f266851a7bfcb0889321999ffc3 not found: ID does not exist"
Sep 18 18:23:41.805237 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.805181    2550 scope.go:117] "RemoveContainer" containerID="1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"
Sep 18 18:23:41.805560 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:23:41.805542    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25\": container with ID starting with 1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25 not found: ID does not exist" containerID="1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"
Sep 18 18:23:41.805630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.805569    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25"} err="failed to get container status \"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25\": rpc error: code = NotFound desc = could not find container \"1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25\": container with ID starting with 1e88e8783227bf1b4f1a34bcc2861c09cfcbb10082bf756e9240961836e5ff25 not found: ID does not exist"
Sep 18 18:23:41.806451 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.806433    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:23:41.810659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:41.810641    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6fa1a2a2-openshift-default-68448cb476-w89st"]
Sep 18 18:23:42.707021 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:42.706993    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4e29416-e22e-483f-9647-233a72bee579" path="/var/lib/kubelet/pods/c4e29416-e22e-483f-9647-233a72bee579/volumes"
Sep 18 18:23:50.991155 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:50.991118    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:23:58.821589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.821546    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"]
Sep 18 18:23:58.822138 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822118    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.822226 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822141    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.822226 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822158    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.822226 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822166    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.822378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822274    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.822378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.822290    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4e29416-e22e-483f-9647-233a72bee579" containerName="istio-proxy"
Sep 18 18:23:58.825449 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.825425    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:23:58.827636 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.827608    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:23:58.827636 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.827618    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:23:58.832171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.832144    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"]
Sep 18 18:23:58.924708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:58.924678    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm\" (UID: \"a3538822-99b5-475a-afe5-dcba81e9cc5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:23:59.026002 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.025977    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm\" (UID: \"a3538822-99b5-475a-afe5-dcba81e9cc5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:23:59.026521 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.026504    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm\" (UID: \"a3538822-99b5-475a-afe5-dcba81e9cc5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:23:59.138420 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.138370    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:23:59.410368 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.410344    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"]
Sep 18 18:23:59.412325 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:23:59.412300    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3538822_99b5_475a_afe5_dcba81e9cc5b.slice/crio-93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07 WatchSource:0}: Error finding container 93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07: Status 404 returned error can't find the container with id 93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07
Sep 18 18:23:59.880749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.880723    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm" event={"ID":"a3538822-99b5-475a-afe5-dcba81e9cc5b","Type":"ContainerStarted","Data":"d650985a8a1a93c820f74d714fc9e7b441360c68ec2428cbd83f151ff513055b"}
Sep 18 18:23:59.881102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.880771    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm" event={"ID":"a3538822-99b5-475a-afe5-dcba81e9cc5b","Type":"ContainerStarted","Data":"93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07"}
Sep 18 18:23:59.897134 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:23:59.897092    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm" podStartSLOduration=1.897078096 podStartE2EDuration="1.897078096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:23:59.895377038 +0000 UTC m=+1411.582006515" watchObservedRunningTime="2026-09-18 18:23:59.897078096 +0000 UTC m=+1411.583707548"
Sep 18 18:24:00.887080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:00.887052    2550 generic.go:356] "Generic (PLEG): container finished" podID="a3538822-99b5-475a-afe5-dcba81e9cc5b" containerID="d650985a8a1a93c820f74d714fc9e7b441360c68ec2428cbd83f151ff513055b" exitCode=0
Sep 18 18:24:00.887492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:00.887126    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm" event={"ID":"a3538822-99b5-475a-afe5-dcba81e9cc5b","Type":"ContainerDied","Data":"d650985a8a1a93c820f74d714fc9e7b441360c68ec2428cbd83f151ff513055b"}
Sep 18 18:24:02.023796 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.023772    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:24:02.049959 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.049217    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca\") pod \"a3538822-99b5-475a-afe5-dcba81e9cc5b\" (UID: \"a3538822-99b5-475a-afe5-dcba81e9cc5b\") "
Sep 18 18:24:02.052026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.050232    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3538822-99b5-475a-afe5-dcba81e9cc5b" (UID: "a3538822-99b5-475a-afe5-dcba81e9cc5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:24:02.151159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.151105    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3538822-99b5-475a-afe5-dcba81e9cc5b-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:02.898413 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.898386    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"
Sep 18 18:24:02.898559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.898429    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm" event={"ID":"a3538822-99b5-475a-afe5-dcba81e9cc5b","Type":"ContainerDied","Data":"93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07"}
Sep 18 18:24:02.898559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:02.898473    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93a6898af555fb85088753ec5da620baea7b6b906e18045d9c1fb41c90097e07"
Sep 18 18:24:03.701719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:03.701689    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:24:03.720496 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:03.720469    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:24:03.728378 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:03.728349    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:24:03.738741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:03.738720    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:24:14.480225 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:14.480147    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"]
Sep 18 18:24:14.485393 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:14.485367    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-37c153-29829255-n2tdh"]
Sep 18 18:24:14.706743 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:14.706709    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5257bc62-9cb0-47b2-b0dd-4d8b8c259386" path="/var/lib/kubelet/pods/5257bc62-9cb0-47b2-b0dd-4d8b8c259386/volumes"
Sep 18 18:24:14.706982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:14.706969    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"]
Sep 18 18:24:14.707034 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:14.706994    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-37c153-h69zm"]
Sep 18 18:24:16.706773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:16.706730    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3538822-99b5-475a-afe5-dcba81e9cc5b" path="/var/lib/kubelet/pods/a3538822-99b5-475a-afe5-dcba81e9cc5b/volumes"
Sep 18 18:24:17.388560 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:17.388515    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:24:18.720954 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.720899    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" podUID="47f344b7-76cf-49b9-9bec-ac155eb5c167" containerName="main" containerID="cri-o://dbc628ae33f4fbfc87eb52f912d46eca5ea0419122cd84d870cad0e19f42aa32" gracePeriod=45
Sep 18 18:24:18.739177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.739141    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" podUID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" containerName="main" containerID="cri-o://231a5768d5d23eda77d108da57164fad377fe1865d442c6725e2d935d21bfafb" gracePeriod=45
Sep 18 18:24:18.746590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.746556    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" podUID="4097882b-92af-442c-acf6-a105ac41a828" containerName="main" containerID="cri-o://d459eca6a4e5b52fb09e36c820164152b27d1a57f9489f0459302ac245753673" gracePeriod=45
Sep 18 18:24:18.756858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.756829    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" podUID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" containerName="main" containerID="cri-o://e32da859fe02e8acc23b129673c6685a7553b57b0e47dc659c40c8d5f299bdb2" gracePeriod=45
Sep 18 18:24:18.986096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.986064    2550 generic.go:356] "Generic (PLEG): container finished" podID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" containerID="e32da859fe02e8acc23b129673c6685a7553b57b0e47dc659c40c8d5f299bdb2" exitCode=0
Sep 18 18:24:18.986228 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.986105    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" event={"ID":"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6","Type":"ContainerDied","Data":"e32da859fe02e8acc23b129673c6685a7553b57b0e47dc659c40c8d5f299bdb2"}
Sep 18 18:24:18.987989 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.987962    2550 generic.go:356] "Generic (PLEG): container finished" podID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" containerID="231a5768d5d23eda77d108da57164fad377fe1865d442c6725e2d935d21bfafb" exitCode=0
Sep 18 18:24:18.988131 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.988106    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" event={"ID":"9ef6eef1-4076-45d5-823e-a3465ba5daa9","Type":"ContainerDied","Data":"231a5768d5d23eda77d108da57164fad377fe1865d442c6725e2d935d21bfafb"}
Sep 18 18:24:18.990276 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.990250    2550 generic.go:356] "Generic (PLEG): container finished" podID="4097882b-92af-442c-acf6-a105ac41a828" containerID="d459eca6a4e5b52fb09e36c820164152b27d1a57f9489f0459302ac245753673" exitCode=0
Sep 18 18:24:18.990366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.990314    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" event={"ID":"4097882b-92af-442c-acf6-a105ac41a828","Type":"ContainerDied","Data":"d459eca6a4e5b52fb09e36c820164152b27d1a57f9489f0459302ac245753673"}
Sep 18 18:24:18.992397 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.992377    2550 generic.go:356] "Generic (PLEG): container finished" podID="47f344b7-76cf-49b9-9bec-ac155eb5c167" containerID="dbc628ae33f4fbfc87eb52f912d46eca5ea0419122cd84d870cad0e19f42aa32" exitCode=0
Sep 18 18:24:18.992491 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:18.992418    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" event={"ID":"47f344b7-76cf-49b9-9bec-ac155eb5c167","Type":"ContainerDied","Data":"dbc628ae33f4fbfc87eb52f912d46eca5ea0419122cd84d870cad0e19f42aa32"}
Sep 18 18:24:19.045017 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.044986    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:24:19.061412 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.061389    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:24:19.068514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.068490    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:24:19.072814 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.072791    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:24:19.189876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189834    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.189876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189882    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l92xp\" (UniqueName: \"kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189906    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg2tp\" (UniqueName: \"kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189924    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189969    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.189986    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190020    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.190154 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190046    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.190461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190291    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home" (OuterVolumeSpecName: "home") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.190528 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190457    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.190528 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190499    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.190637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190454    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache" (OuterVolumeSpecName: "model-cache") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.190637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190531    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w9cg\" (UniqueName: \"kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.190637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190599    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.190807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190640    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.190807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190663    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.190807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190741    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.190807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190769    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190807    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache\") pod \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\" (UID: \"9ef6eef1-4076-45d5-823e-a3465ba5daa9\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190843    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190867    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190895    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190955    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs\") pod \"4097882b-92af-442c-acf6-a105ac41a828\" (UID: \"4097882b-92af-442c-acf6-a105ac41a828\") "
Sep 18 18:24:19.191037 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.190988    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.191347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.191045    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm\") pod \"47f344b7-76cf-49b9-9bec-ac155eb5c167\" (UID: \"47f344b7-76cf-49b9-9bec-ac155eb5c167\") "
Sep 18 18:24:19.191347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.191091    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md7zv\" (UniqueName: \"kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv\") pod \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\" (UID: \"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6\") "
Sep 18 18:24:19.191492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.191470    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.191553 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.191496    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.192756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192719    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp" (OuterVolumeSpecName: "kube-api-access-jg2tp") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "kube-api-access-jg2tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:19.192756 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192732    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm" (OuterVolumeSpecName: "dshm") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192781    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:19.193110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192963    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192989    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp" (OuterVolumeSpecName: "kube-api-access-l92xp") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "kube-api-access-l92xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:19.193110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.192998    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:19.193353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193184    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193207    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache" (OuterVolumeSpecName: "model-cache") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193517 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193358    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home" (OuterVolumeSpecName: "home") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193517 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193364    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache" (OuterVolumeSpecName: "model-cache") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193517 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193481    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm" (OuterVolumeSpecName: "dshm") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193681    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193696    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home" (OuterVolumeSpecName: "home") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193821 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193776    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv" (OuterVolumeSpecName: "kube-api-access-md7zv") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "kube-api-access-md7zv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:19.193880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193806    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg" (OuterVolumeSpecName: "kube-api-access-7w9cg") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "kube-api-access-7w9cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:24:19.193960 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193883    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.193960 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.193890    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ef6eef1-4076-45d5-823e-a3465ba5daa9" (UID: "9ef6eef1-4076-45d5-823e-a3465ba5daa9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.194067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.194022    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home" (OuterVolumeSpecName: "home") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.194905 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.194887    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm" (OuterVolumeSpecName: "dshm") pod "47f344b7-76cf-49b9-9bec-ac155eb5c167" (UID: "47f344b7-76cf-49b9-9bec-ac155eb5c167"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.195018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.195001    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" (UID: "da58b3dd-9c49-4e42-a945-b6e5cfa1feb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:19.195327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.195309    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm" (OuterVolumeSpecName: "dshm") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:24:19.195391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.195379    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4097882b-92af-442c-acf6-a105ac41a828" (UID: "4097882b-92af-442c-acf6-a105ac41a828"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:24:19.292671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292649    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/47f344b7-76cf-49b9-9bec-ac155eb5c167-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292671 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292669    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292677    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292685    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w9cg\" (UniqueName: \"kubernetes.io/projected/9ef6eef1-4076-45d5-823e-a3465ba5daa9-kube-api-access-7w9cg\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292695    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292703    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292710    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292717    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4097882b-92af-442c-acf6-a105ac41a828-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292723    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292730    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292737    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292744    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292750    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292756    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4097882b-92af-442c-acf6-a105ac41a828-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292763    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292770    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/47f344b7-76cf-49b9-9bec-ac155eb5c167-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292777    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md7zv\" (UniqueName: \"kubernetes.io/projected/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-kube-api-access-md7zv\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292785    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l92xp\" (UniqueName: \"kubernetes.io/projected/47f344b7-76cf-49b9-9bec-ac155eb5c167-kube-api-access-l92xp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292795    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg2tp\" (UniqueName: \"kubernetes.io/projected/4097882b-92af-442c-acf6-a105ac41a828-kube-api-access-jg2tp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292803    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.292804 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292811    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef6eef1-4076-45d5-823e-a3465ba5daa9-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.293384 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.292818    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef6eef1-4076-45d5-823e-a3465ba5daa9-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:24:19.999399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.999332    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"
Sep 18 18:24:19.999399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.999338    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q" event={"ID":"4097882b-92af-442c-acf6-a105ac41a828","Type":"ContainerDied","Data":"2ba666a66bd112b436ff7700bfd547b8d28269552c75ebdfadb7fa31fe502b98"}
Sep 18 18:24:19.999813 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:19.999401    2550 scope.go:117] "RemoveContainer" containerID="d459eca6a4e5b52fb09e36c820164152b27d1a57f9489f0459302ac245753673"
Sep 18 18:24:20.001065 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.001041    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7" event={"ID":"47f344b7-76cf-49b9-9bec-ac155eb5c167","Type":"ContainerDied","Data":"fdb4cc911ba3a83d10e75b3e93a5ace3ce93ab424c281ddf9607791ce17be9c1"}
Sep 18 18:24:20.001194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.001051    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"
Sep 18 18:24:20.002778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.002755    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"
Sep 18 18:24:20.002892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.002748    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q" event={"ID":"da58b3dd-9c49-4e42-a945-b6e5cfa1feb6","Type":"ContainerDied","Data":"4dcff2298631f011367dbf74bdc9af1f70f5ff0bc437951141f5186a2d29cf85"}
Sep 18 18:24:20.004319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.004294    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt" event={"ID":"9ef6eef1-4076-45d5-823e-a3465ba5daa9","Type":"ContainerDied","Data":"52b1563fb500eacd02201c7002db8d205c9caf5585c0fb3f3dc928b5e7453777"}
Sep 18 18:24:20.004319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.004307    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"
Sep 18 18:24:20.010117 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.010094    2550 scope.go:117] "RemoveContainer" containerID="dbc628ae33f4fbfc87eb52f912d46eca5ea0419122cd84d870cad0e19f42aa32"
Sep 18 18:24:20.019809 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.019795    2550 scope.go:117] "RemoveContainer" containerID="e32da859fe02e8acc23b129673c6685a7553b57b0e47dc659c40c8d5f299bdb2"
Sep 18 18:24:20.027876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.027848    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:24:20.029415 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.029400    2550 scope.go:117] "RemoveContainer" containerID="231a5768d5d23eda77d108da57164fad377fe1865d442c6725e2d935d21bfafb"
Sep 18 18:24:20.031200 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.031178    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/premium-w0-37c153-kserve-79bb469d6b-f2j7q"]
Sep 18 18:24:20.040450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.040430    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:24:20.044020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.043999    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/distinct-2-w0-37c153-kserve-7cb65547f4-jknk7"]
Sep 18 18:24:20.055297 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.055279    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:24:20.058568 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.058547    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/embedding-w0-37c153-kserve-f89c586-v2wzt"]
Sep 18 18:24:20.069096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.069078    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:24:20.072736 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.072719    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-37c153/simulator-w0-37c153-kserve-85544d9bc-vh97q"]
Sep 18 18:24:20.706926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.706895    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4097882b-92af-442c-acf6-a105ac41a828" path="/var/lib/kubelet/pods/4097882b-92af-442c-acf6-a105ac41a828/volumes"
Sep 18 18:24:20.707305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.707288    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47f344b7-76cf-49b9-9bec-ac155eb5c167" path="/var/lib/kubelet/pods/47f344b7-76cf-49b9-9bec-ac155eb5c167/volumes"
Sep 18 18:24:20.707642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.707630    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" path="/var/lib/kubelet/pods/9ef6eef1-4076-45d5-823e-a3465ba5daa9/volumes"
Sep 18 18:24:20.708279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:20.708263    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" path="/var/lib/kubelet/pods/da58b3dd-9c49-4e42-a945-b6e5cfa1feb6/volumes"
Sep 18 18:24:30.497675 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.497643    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498049    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3538822-99b5-475a-afe5-dcba81e9cc5b" containerName="revoke-keys"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498062    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3538822-99b5-475a-afe5-dcba81e9cc5b" containerName="revoke-keys"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498072    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498078    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498094    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47f344b7-76cf-49b9-9bec-ac155eb5c167" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498099    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="47f344b7-76cf-49b9-9bec-ac155eb5c167" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498108    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498113    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498120    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4097882b-92af-442c-acf6-a105ac41a828" containerName="main"
Sep 18 18:24:30.498176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498126    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="4097882b-92af-442c-acf6-a105ac41a828" containerName="main"
Sep 18 18:24:30.498525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498192    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="47f344b7-76cf-49b9-9bec-ac155eb5c167" containerName="main"
Sep 18 18:24:30.498525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498200    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="4097882b-92af-442c-acf6-a105ac41a828" containerName="main"
Sep 18 18:24:30.498525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498207    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3538822-99b5-475a-afe5-dcba81e9cc5b" containerName="revoke-keys"
Sep 18 18:24:30.498525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498214    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef6eef1-4076-45d5-823e-a3465ba5daa9" containerName="main"
Sep 18 18:24:30.498525 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.498220    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="da58b3dd-9c49-4e42-a945-b6e5cfa1feb6" containerName="main"
Sep 18 18:24:30.502839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.502818    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.505223 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.505201    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e6108f-openshift-default-dockercfg-shkbd\""
Sep 18 18:24:30.514998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.514978    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:24:30.588466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588436    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588471    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588495    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588516    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588571    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzm2m\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588607    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588637    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.588672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588654    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.589066 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.588718    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.689890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.689858    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.689890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.689891    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.689913    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.689932    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.689984    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzm2m\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690002    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690027    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690073    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690439 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690149    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690439 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690320    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690558 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690521    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690558 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690544    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690581    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.690678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.690669    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.692758 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.692737    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.693086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.693067    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.698184 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.698159    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.698276 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.698167    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzm2m\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m\") pod \"e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") " pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.815348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.815321    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-e6108f-openshift-default-dockercfg-shkbd\""
Sep 18 18:24:30.824029 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.824010    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:30.902634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:30.902604    2550 scope.go:117] "RemoveContainer" containerID="9750ce8ec6695010dce1b886be8353d53c2d6d7d240fd8aff609478f104583ab"
Sep 18 18:24:31.099135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:31.099111    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:24:31.101278 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:24:31.101240    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8752b1fd_af38_4aec_bf36_570e45ac65bd.slice/crio-50bfdf9d4f0cc99508b4c59ef96c8e495a421afd85152d3564586e1ea73e11d8 WatchSource:0}: Error finding container 50bfdf9d4f0cc99508b4c59ef96c8e495a421afd85152d3564586e1ea73e11d8: Status 404 returned error can't find the container with id 50bfdf9d4f0cc99508b4c59ef96c8e495a421afd85152d3564586e1ea73e11d8
Sep 18 18:24:31.103372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:31.103344    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:31.103464 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:31.103401    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:31.103464 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:31.103438    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:32.077827 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:32.077792    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerStarted","Data":"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"}
Sep 18 18:24:32.078302 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:32.077847    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerStarted","Data":"50bfdf9d4f0cc99508b4c59ef96c8e495a421afd85152d3564586e1ea73e11d8"}
Sep 18 18:24:32.099343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:32.099296    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podStartSLOduration=2.099281382 podStartE2EDuration="2.099281382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:24:32.097677788 +0000 UTC m=+1443.784307241" watchObservedRunningTime="2026-09-18 18:24:32.099281382 +0000 UTC m=+1443.785910834"
Sep 18 18:24:32.824232 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:32.824197    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:32.829440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:32.829415    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:33.083232 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:33.083152    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:33.084048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:33.084029    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:37.810612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.810576    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:24:37.814641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.814621    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.816959 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.816907    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-e6108f-dockercfg-ghkln\""
Sep 18 18:24:37.817079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.816924    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-e6108f\""
Sep 18 18:24:37.820960 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.820915    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:24:37.859711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.859681    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.859842 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.859750    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqq5b\" (UniqueName: \"kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.960881 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.960848    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqq5b\" (UniqueName: \"kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.961059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.961001    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.961668 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.961648    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:37.969692 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:37.969664    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqq5b\" (UniqueName: \"kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b\") pod \"payload-processing-e2e-api-a-e6108f-977b55475-7hjwj\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") " pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:38.126544 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:38.126486    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:38.397124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:38.397098    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:24:38.399423 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:24:38.399399    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode154ae6c_adf2_4546_a3f4_b8e556f0e546.slice/crio-f740359cb9dd53240e960305f72fcde539f07e24af3e592328410c7750e13677 WatchSource:0}: Error finding container f740359cb9dd53240e960305f72fcde539f07e24af3e592328410c7750e13677: Status 404 returned error can't find the container with id f740359cb9dd53240e960305f72fcde539f07e24af3e592328410c7750e13677
Sep 18 18:24:39.114678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:39.114639    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" event={"ID":"e154ae6c-adf2-4546-a3f4-b8e556f0e546","Type":"ContainerStarted","Data":"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"}
Sep 18 18:24:39.115147 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:39.114697    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" event={"ID":"e154ae6c-adf2-4546-a3f4-b8e556f0e546","Type":"ContainerStarted","Data":"f740359cb9dd53240e960305f72fcde539f07e24af3e592328410c7750e13677"}
Sep 18 18:24:39.115147 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:39.114790    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:39.131589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:39.131545    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" podStartSLOduration=2.131533101 podStartE2EDuration="2.131533101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:24:39.129794853 +0000 UTC m=+1450.816424320" watchObservedRunningTime="2026-09-18 18:24:39.131533101 +0000 UTC m=+1450.818162544"
Sep 18 18:24:41.084120 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:41.084086    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:24:49.125172 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:49.125117    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 18:24:49.125711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:49.125183    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 18:24:50.122894 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:50.122860    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:24:51.189737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.189702    2550 generic.go:356] "Generic (PLEG): container finished" podID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerID="1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270" exitCode=0
Sep 18 18:24:51.190197 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.189771    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerDied","Data":"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"}
Sep 18 18:24:51.190197 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.190145    2550 scope.go:117] "RemoveContainer" containerID="1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"
Sep 18 18:24:51.191228 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.191202    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:51.191296 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.191268    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:51.191335 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:51.191298    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:24:52.198204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:52.198165    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerStarted","Data":"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"}
Sep 18 18:24:52.825193 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:52.825153    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:52.825385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:52.825363    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 18:24:52.825433 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:52.825413    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 18:24:53.825002 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:53.824971    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 18:24:53.825404 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:53.825026    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 18:24:54.824576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:54.824540    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 18:24:54.824760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:54.824598    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 18:24:55.824583 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:55.824551    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 18:24:55.825091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:55.824614    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 18:24:56.825363 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:56.825326    2550 patch_prober.go:28] interesting pod/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Sep 18 18:24:56.825783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:56.825385    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Sep 18 18:24:57.828477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:57.828446    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:57.828909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:57.828723    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:24:57.829503 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:24:57.829487    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:25:00.077960 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:00.077906    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:25:12.602019 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.601984    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:25:12.605535 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.605512    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.607678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.607657    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-29f0cb\""
Sep 18 18:25:12.607768 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.607676    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-29f0cb-dockercfg-x7xpr\""
Sep 18 18:25:12.613628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.613607    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:25:12.778690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.778638    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgmms\" (UniqueName: \"kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.778890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.778754    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.879863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.879766    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgmms\" (UniqueName: \"kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.879863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.879838    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.880532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.880512    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.887557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.887529    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgmms\" (UniqueName: \"kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms\") pod \"payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") " pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:12.916106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:12.916079    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:13.187724 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:13.187699    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:25:13.189499 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:25:13.189468    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f19fe0_f794_447c_80e3_f7dda3cc28e9.slice/crio-a6ec8907051ba2c2e1e5bd1a6769ccffb9cbe670f8a2a712937b1984f5ec8e7c WatchSource:0}: Error finding container a6ec8907051ba2c2e1e5bd1a6769ccffb9cbe670f8a2a712937b1984f5ec8e7c: Status 404 returned error can't find the container with id a6ec8907051ba2c2e1e5bd1a6769ccffb9cbe670f8a2a712937b1984f5ec8e7c
Sep 18 18:25:13.321880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:13.321842    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" event={"ID":"85f19fe0-f794-447c-80e3-f7dda3cc28e9","Type":"ContainerStarted","Data":"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"}
Sep 18 18:25:13.322080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:13.321946    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:13.322080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:13.321974    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" event={"ID":"85f19fe0-f794-447c-80e3-f7dda3cc28e9","Type":"ContainerStarted","Data":"a6ec8907051ba2c2e1e5bd1a6769ccffb9cbe670f8a2a712937b1984f5ec8e7c"}
Sep 18 18:25:13.338952 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:13.338890    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" podStartSLOduration=1.338877016 podStartE2EDuration="1.338877016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:25:13.337326218 +0000 UTC m=+1485.023955703" watchObservedRunningTime="2026-09-18 18:25:13.338877016 +0000 UTC m=+1485.025506484"
Sep 18 18:25:17.149011 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:17.148979    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:25:18.162588 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:18.162554    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:25:22.678309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:22.678264    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:25:23.681285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.681252    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:25:23.684329 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.684308    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.686589 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.686567    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-011541\"/\"sub-test-model-011541-kserve-self-signed-certs\""
Sep 18 18:25:23.695155 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.695132    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:25:23.871668 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.871635    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.871819 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.871676    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.871819 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.871693    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.871819 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.871803    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.871998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.871979    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wprq\" (UniqueName: \"kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.872048 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.872011    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.972984    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973039    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wprq\" (UniqueName: \"kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973159    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973210    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973261    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973424 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973287    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973582 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973543    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973716 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973599    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.973716 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.973630    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.975437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.975414    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.975514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.975489    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.982323 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.982302    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wprq\" (UniqueName: \"kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq\") pod \"sub-test-model-011541-kserve-84dbf65459-vs5jx\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") " pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:23.994170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:23.994147    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:24.266745 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:24.266714    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:25:24.270898 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:25:24.270871    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb357868a_6f1f_4c0d_b73e_85fa666f8a2d.slice/crio-8d16cab4eda49f6ed138a229068b52e45701ceb49f8e8ace8e4f10fbabc1e152 WatchSource:0}: Error finding container 8d16cab4eda49f6ed138a229068b52e45701ceb49f8e8ace8e4f10fbabc1e152: Status 404 returned error can't find the container with id 8d16cab4eda49f6ed138a229068b52e45701ceb49f8e8ace8e4f10fbabc1e152
Sep 18 18:25:24.329274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:24.329253    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:25:24.387251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:24.387218    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" event={"ID":"b357868a-6f1f-4c0d-b73e-85fa666f8a2d","Type":"ContainerStarted","Data":"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"}
Sep 18 18:25:24.387398 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:24.387260    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" event={"ID":"b357868a-6f1f-4c0d-b73e-85fa666f8a2d","Type":"ContainerStarted","Data":"8d16cab4eda49f6ed138a229068b52e45701ceb49f8e8ace8e4f10fbabc1e152"}
Sep 18 18:25:24.404601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:24.404448    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" podStartSLOduration=1.404428258 podStartE2EDuration="1.404428258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:25:24.403837067 +0000 UTC m=+1496.090466558" watchObservedRunningTime="2026-09-18 18:25:24.404428258 +0000 UTC m=+1496.091057713"
Sep 18 18:25:30.775053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:30.775023    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:25:30.779077 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:30.779056    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:25:30.939231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:30.939204    2550 scope.go:117] "RemoveContainer" containerID="17609e3edfe3cb272598b6e5cc60911c31dba8ed01f875f073de8b6814dddf33"
Sep 18 18:25:32.167717 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.167678    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerName="main" containerID="cri-o://d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328" gracePeriod=45
Sep 18 18:25:32.237385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.237350    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": dial tcp 10.133.0.70:8000: connect: connection refused"
Sep 18 18:25:32.416784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.416761    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:25:32.430465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.430415    2550 generic.go:356] "Generic (PLEG): container finished" podID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerID="d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328" exitCode=0
Sep 18 18:25:32.430465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.430459    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"
Sep 18 18:25:32.430594 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.430501    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" event={"ID":"a87885eb-8f12-4795-b251-e45129ba1ab8","Type":"ContainerDied","Data":"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"}
Sep 18 18:25:32.430594 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.430543    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz" event={"ID":"a87885eb-8f12-4795-b251-e45129ba1ab8","Type":"ContainerDied","Data":"95fb1838114b922eaded7836c6e0f5f620f80cad3dd66832dbcc10181a2c17b5"}
Sep 18 18:25:32.430594 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.430566    2550 scope.go:117] "RemoveContainer" containerID="d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"
Sep 18 18:25:32.441015 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.440930    2550 scope.go:117] "RemoveContainer" containerID="d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"
Sep 18 18:25:32.441312 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:25:32.441278    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328\": container with ID starting with d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328 not found: ID does not exist" containerID="d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"
Sep 18 18:25:32.441390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.441313    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328"} err="failed to get container status \"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328\": rpc error: code = NotFound desc = could not find container \"d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328\": container with ID starting with d21acf4a61f6a42b37aeafff3868d25b969ffa4aa654e2512d3f9d1aaade4328 not found: ID does not exist"
Sep 18 18:25:32.443675 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443656    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rxht\" (UniqueName: \"kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.443771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443713    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.443831 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443786    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.443831 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443815    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.443952 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443837    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.443952 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.443865    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home\") pod \"a87885eb-8f12-4795-b251-e45129ba1ab8\" (UID: \"a87885eb-8f12-4795-b251-e45129ba1ab8\") "
Sep 18 18:25:32.444202 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.444146    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:32.444313 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.444235    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:32.444373 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.444311    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home" (OuterVolumeSpecName: "home") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:32.445903 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.445874    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht" (OuterVolumeSpecName: "kube-api-access-9rxht") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "kube-api-access-9rxht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:25:32.446189 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.446169    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:25:32.446258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.446194    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm" (OuterVolumeSpecName: "dshm") pod "a87885eb-8f12-4795-b251-e45129ba1ab8" (UID: "a87885eb-8f12-4795-b251-e45129ba1ab8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:32.544912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544892    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.544912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544911    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.545059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544920    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a87885eb-8f12-4795-b251-e45129ba1ab8-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.545059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544929    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.545059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544957    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a87885eb-8f12-4795-b251-e45129ba1ab8-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.545059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.544966    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rxht\" (UniqueName: \"kubernetes.io/projected/a87885eb-8f12-4795-b251-e45129ba1ab8-kube-api-access-9rxht\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:32.745274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.745248    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:25:32.751038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:32.751017    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/auth-test-model-011541-kserve-55989c4ff5-kr5jz"]
Sep 18 18:25:33.182446 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.182410    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" podUID="0f334b50-f913-42f1-a32c-ff5e12335b45" containerName="main" containerID="cri-o://bcbd06c2c10054e5cdda2be1e44a81999af2056622fde0a70be0db272d13f0a7" gracePeriod=45
Sep 18 18:25:33.437486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.437413    2550 generic.go:356] "Generic (PLEG): container finished" podID="0f334b50-f913-42f1-a32c-ff5e12335b45" containerID="bcbd06c2c10054e5cdda2be1e44a81999af2056622fde0a70be0db272d13f0a7" exitCode=0
Sep 18 18:25:33.437623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.437485    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" event={"ID":"0f334b50-f913-42f1-a32c-ff5e12335b45","Type":"ContainerDied","Data":"bcbd06c2c10054e5cdda2be1e44a81999af2056622fde0a70be0db272d13f0a7"}
Sep 18 18:25:33.437623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.437523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j" event={"ID":"0f334b50-f913-42f1-a32c-ff5e12335b45","Type":"ContainerDied","Data":"a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46"}
Sep 18 18:25:33.437623 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.437534    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a5f7837c282b37adffb4b99b327ebcda8d1bd6dff487888e13df3bc4ceb74a46"
Sep 18 18:25:33.448285 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.448266    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:25:33.554505 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554468    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554505 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554517    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554549    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554578    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fmcp\" (UniqueName: \"kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554604    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554674    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home\") pod \"0f334b50-f913-42f1-a32c-ff5e12335b45\" (UID: \"0f334b50-f913-42f1-a32c-ff5e12335b45\") "
Sep 18 18:25:33.554868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554752    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache" (OuterVolumeSpecName: "model-cache") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:33.554868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554806    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:33.555015 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.554993    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home" (OuterVolumeSpecName: "home") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:33.555085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.555000    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.555085 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.555046    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.556653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.556626    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm" (OuterVolumeSpecName: "dshm") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:25:33.556775 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.556738    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp" (OuterVolumeSpecName: "kube-api-access-7fmcp") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "kube-api-access-7fmcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:25:33.556839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.556789    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f334b50-f913-42f1-a32c-ff5e12335b45" (UID: "0f334b50-f913-42f1-a32c-ff5e12335b45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:25:33.656166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.656134    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.656166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.656164    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f334b50-f913-42f1-a32c-ff5e12335b45-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.656166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.656173    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmcp\" (UniqueName: \"kubernetes.io/projected/0f334b50-f913-42f1-a32c-ff5e12335b45-kube-api-access-7fmcp\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.656347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.656181    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f334b50-f913-42f1-a32c-ff5e12335b45-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:33.995243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.995214    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:33.995426 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:33.995260    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:34.007492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.007466    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:34.443510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.443481    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"
Sep 18 18:25:34.464694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.459391    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:25:34.472514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.472488    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:25:34.474293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.474270    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/auth-test-model-99602c-kserve-6c99db86c5-n5m9j"]
Sep 18 18:25:34.707141 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.707063    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f334b50-f913-42f1-a32c-ff5e12335b45" path="/var/lib/kubelet/pods/0f334b50-f913-42f1-a32c-ff5e12335b45/volumes"
Sep 18 18:25:34.707476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:34.707462    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" path="/var/lib/kubelet/pods/a87885eb-8f12-4795-b251-e45129ba1ab8/volumes"
Sep 18 18:25:41.080231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.080195    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"]
Sep 18 18:25:41.081134 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081107    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f334b50-f913-42f1-a32c-ff5e12335b45" containerName="main"
Sep 18 18:25:41.081134 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081136    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f334b50-f913-42f1-a32c-ff5e12335b45" containerName="main"
Sep 18 18:25:41.081334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081153    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerName="main"
Sep 18 18:25:41.081334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081163    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerName="main"
Sep 18 18:25:41.081438 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081359    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a87885eb-8f12-4795-b251-e45129ba1ab8" containerName="main"
Sep 18 18:25:41.081438 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.081378    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f334b50-f913-42f1-a32c-ff5e12335b45" containerName="main"
Sep 18 18:25:41.086417 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.086396    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:41.090317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.090293    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"]
Sep 18 18:25:41.091531 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.091500    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:25:41.091760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.091744    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:25:41.120848 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.120828    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2\" (UID: \"96b61596-a827-49a7-92a3-12913cf38677\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:41.221216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.221193    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2\" (UID: \"96b61596-a827-49a7-92a3-12913cf38677\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:41.221738 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.221720    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2\" (UID: \"96b61596-a827-49a7-92a3-12913cf38677\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:41.398235 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.398187    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:41.668703 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:41.668677    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"]
Sep 18 18:25:41.671180 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:25:41.671144    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b61596_a827_49a7_92a3_12913cf38677.slice/crio-825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159 WatchSource:0}: Error finding container 825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159: Status 404 returned error can't find the container with id 825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159
Sep 18 18:25:42.488631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:42.488605    2550 generic.go:356] "Generic (PLEG): container finished" podID="96b61596-a827-49a7-92a3-12913cf38677" containerID="df3ba59e049f49428742b458ac27c18c004d2224d57dae608af154cbfb271641" exitCode=0
Sep 18 18:25:42.489010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:42.488699    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2" event={"ID":"96b61596-a827-49a7-92a3-12913cf38677","Type":"ContainerDied","Data":"df3ba59e049f49428742b458ac27c18c004d2224d57dae608af154cbfb271641"}
Sep 18 18:25:42.489010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:42.488751    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2" event={"ID":"96b61596-a827-49a7-92a3-12913cf38677","Type":"ContainerStarted","Data":"825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159"}
Sep 18 18:25:43.184838 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:43.184803    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:25:43.637043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:43.637004    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:43.740312 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:43.740283    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca\") pod \"96b61596-a827-49a7-92a3-12913cf38677\" (UID: \"96b61596-a827-49a7-92a3-12913cf38677\") "
Sep 18 18:25:43.740704 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:43.740676    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96b61596-a827-49a7-92a3-12913cf38677" (UID: "96b61596-a827-49a7-92a3-12913cf38677"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:25:43.841416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:43.841384    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96b61596-a827-49a7-92a3-12913cf38677-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:44.091328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.091300    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:25:44.091731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.091694    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b61596-a827-49a7-92a3-12913cf38677" containerName="revoke-keys"
Sep 18 18:25:44.091731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.091710    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b61596-a827-49a7-92a3-12913cf38677" containerName="revoke-keys"
Sep 18 18:25:44.091817 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.091799    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b61596-a827-49a7-92a3-12913cf38677" containerName="revoke-keys"
Sep 18 18:25:44.094952 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.094924    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.098268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.098247    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"default-dockercfg-r7zmk\""
Sep 18 18:25:44.098445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.098247    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"openshift-service-ca.crt\""
Sep 18 18:25:44.098526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.098251    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"sub-test-model-99602c-kserve-self-signed-certs\""
Sep 18 18:25:44.098526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.098255    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"kube-root-ca.crt\""
Sep 18 18:25:44.105130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.105106    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:25:44.144380 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144359    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.144476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144401    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.144476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144432    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.144554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144471    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.144554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144495    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.144554 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.144526    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f64th\" (UniqueName: \"kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.245780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245756    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.245888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245797    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.245888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245827    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.245888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245855    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.245888 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245879    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.246111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.245915    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f64th\" (UniqueName: \"kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.246201 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.246169    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.246264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.246235    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.246467 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.246439    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.248133 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.248110    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.248420 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.248404    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.255149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.255130    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f64th\" (UniqueName: \"kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th\") pod \"sub-test-model-99602c-kserve-7fc9c48457-zfl2r\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") " pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.407143 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.407073    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:44.501177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.501150    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"
Sep 18 18:25:44.501177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.501167    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2" event={"ID":"96b61596-a827-49a7-92a3-12913cf38677","Type":"ContainerDied","Data":"825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159"}
Sep 18 18:25:44.501329 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.501210    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="825cf71bc4c48af120217c10f6706f6f00c9a43f4a8400443287e98d7be75159"
Sep 18 18:25:44.677702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:44.677642    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:25:44.680396 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:25:44.680361    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2808838_f633_44ca_80d7_f474857033c9.slice/crio-9999cd0d23a6e1c6c601ecb4781bfb70722df315ed6ac64c569ea3773f350010 WatchSource:0}: Error finding container 9999cd0d23a6e1c6c601ecb4781bfb70722df315ed6ac64c569ea3773f350010: Status 404 returned error can't find the container with id 9999cd0d23a6e1c6c601ecb4781bfb70722df315ed6ac64c569ea3773f350010
Sep 18 18:25:45.512010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:45.511979    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" event={"ID":"e2808838-f633-44ca-80d7-f474857033c9","Type":"ContainerStarted","Data":"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"}
Sep 18 18:25:45.512010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:45.512017    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" event={"ID":"e2808838-f633-44ca-80d7-f474857033c9","Type":"ContainerStarted","Data":"9999cd0d23a6e1c6c601ecb4781bfb70722df315ed6ac64c569ea3773f350010"}
Sep 18 18:25:45.529633 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:45.529581    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" podStartSLOduration=1.529557553 podStartE2EDuration="1.529557553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:25:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:25:45.528399851 +0000 UTC m=+1517.215029305" watchObservedRunningTime="2026-09-18 18:25:45.529557553 +0000 UTC m=+1517.216187001"
Sep 18 18:25:54.407676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:54.407643    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:54.407676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:54.407680    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:54.420432 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:54.420407    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:54.570896 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:54.570866    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:25:56.778811 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:56.778774    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:25:56.993798 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:56.993765    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:25:56.994027 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:56.993998    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" podUID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" containerName="payload-processing" containerID="cri-o://0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241" gracePeriod=30
Sep 18 18:25:57.188472 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.188434    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"]
Sep 18 18:25:57.188708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.188689    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e6108f-nwbs2"]
Sep 18 18:25:57.245986 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.245927    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:25:57.357643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.357542    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume\") pod \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") "
Sep 18 18:25:57.357643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.357622    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqq5b\" (UniqueName: \"kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b\") pod \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\" (UID: \"e154ae6c-adf2-4546-a3f4-b8e556f0e546\") "
Sep 18 18:25:57.357912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.357889    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e154ae6c-adf2-4546-a3f4-b8e556f0e546" (UID: "e154ae6c-adf2-4546-a3f4-b8e556f0e546"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:25:57.359844 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.359820    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b" (OuterVolumeSpecName: "kube-api-access-vqq5b") pod "e154ae6c-adf2-4546-a3f4-b8e556f0e546" (UID: "e154ae6c-adf2-4546-a3f4-b8e556f0e546"). InnerVolumeSpecName "kube-api-access-vqq5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:25:57.459239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.459198    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e154ae6c-adf2-4546-a3f4-b8e556f0e546-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:57.459239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.459229    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqq5b\" (UniqueName: \"kubernetes.io/projected/e154ae6c-adf2-4546-a3f4-b8e556f0e546-kube-api-access-vqq5b\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:25:57.577736 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.577699    2550 generic.go:356] "Generic (PLEG): container finished" podID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" containerID="0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241" exitCode=0
Sep 18 18:25:57.577901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.577744    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"
Sep 18 18:25:57.577901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.577785    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" event={"ID":"e154ae6c-adf2-4546-a3f4-b8e556f0e546","Type":"ContainerDied","Data":"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"}
Sep 18 18:25:57.577901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.577869    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj" event={"ID":"e154ae6c-adf2-4546-a3f4-b8e556f0e546","Type":"ContainerDied","Data":"f740359cb9dd53240e960305f72fcde539f07e24af3e592328410c7750e13677"}
Sep 18 18:25:57.577901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.577892    2550 scope.go:117] "RemoveContainer" containerID="0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"
Sep 18 18:25:57.587228 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.587209    2550 scope.go:117] "RemoveContainer" containerID="0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"
Sep 18 18:25:57.587477 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:25:57.587458    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241\": container with ID starting with 0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241 not found: ID does not exist" containerID="0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"
Sep 18 18:25:57.587526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.587484    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241"} err="failed to get container status \"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241\": rpc error: code = NotFound desc = could not find container \"0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241\": container with ID starting with 0f93d2d2a46e0922f194cc0292c91cf06cc2e57fd839d9ee920dcd3bed0e5241 not found: ID does not exist"
Sep 18 18:25:57.600924 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.600900    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:25:57.604559 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:57.604538    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e6108f-977b55475-7hjwj"]
Sep 18 18:25:58.706604 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:58.706570    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b61596-a827-49a7-92a3-12913cf38677" path="/var/lib/kubelet/pods/96b61596-a827-49a7-92a3-12913cf38677/volumes"
Sep 18 18:25:58.707007 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:25:58.706930    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" path="/var/lib/kubelet/pods/e154ae6c-adf2-4546-a3f4-b8e556f0e546/volumes"
Sep 18 18:26:03.850469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:03.850428    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:26:03.850966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:03.850799    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy" containerID="cri-o://2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825" gracePeriod=30
Sep 18 18:26:04.239434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.239358    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"]
Sep 18 18:26:04.239741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.239728    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" containerName="payload-processing"
Sep 18 18:26:04.239793 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.239743    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" containerName="payload-processing"
Sep 18 18:26:04.239836 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.239817    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e154ae6c-adf2-4546-a3f4-b8e556f0e546" containerName="payload-processing"
Sep 18 18:26:04.244309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.244291    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:04.246563 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.246534    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:26:04.246743 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.246600    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:26:04.249779 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.249755    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"]
Sep 18 18:26:04.327516 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.327482    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4\" (UID: \"8e04c2a7-e60e-421d-8675-e50a59d600dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:04.428931 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.428898    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4\" (UID: \"8e04c2a7-e60e-421d-8675-e50a59d600dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:04.429498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.429478    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4\" (UID: \"8e04c2a7-e60e-421d-8675-e50a59d600dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:04.556423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.556377    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:04.828148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:04.828118    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"]
Sep 18 18:26:04.830598 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:26:04.830573    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e04c2a7_e60e_421d_8675_e50a59d600dd.slice/crio-4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027 WatchSource:0}: Error finding container 4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027: Status 404 returned error can't find the container with id 4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027
Sep 18 18:26:05.623268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:05.623241    2550 generic.go:356] "Generic (PLEG): container finished" podID="8e04c2a7-e60e-421d-8675-e50a59d600dd" containerID="6c3012940e21b92862697ed13da7f08c19e918a4ace7cf9b9fb3b96bd867bf7f" exitCode=0
Sep 18 18:26:05.623627 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:05.623327    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4" event={"ID":"8e04c2a7-e60e-421d-8675-e50a59d600dd","Type":"ContainerDied","Data":"6c3012940e21b92862697ed13da7f08c19e918a4ace7cf9b9fb3b96bd867bf7f"}
Sep 18 18:26:05.623627 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:05.623377    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4" event={"ID":"8e04c2a7-e60e-421d-8675-e50a59d600dd","Type":"ContainerStarted","Data":"4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027"}
Sep 18 18:26:06.765332 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:06.765306    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:06.850571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:06.850547    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca\") pod \"8e04c2a7-e60e-421d-8675-e50a59d600dd\" (UID: \"8e04c2a7-e60e-421d-8675-e50a59d600dd\") "
Sep 18 18:26:06.850876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:06.850855    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e04c2a7-e60e-421d-8675-e50a59d600dd" (UID: "8e04c2a7-e60e-421d-8675-e50a59d600dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:06.952051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:06.952003    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e04c2a7-e60e-421d-8675-e50a59d600dd-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:07.635476 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:07.635447    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"
Sep 18 18:26:07.635651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:07.635497    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4" event={"ID":"8e04c2a7-e60e-421d-8675-e50a59d600dd","Type":"ContainerDied","Data":"4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027"}
Sep 18 18:26:07.635651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:07.635547    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f4abaea4a52e37188d73d4e08102f24aade8cfb2e1cca7b40216c304e944027"
Sep 18 18:26:09.098590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.098567    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:26:09.268759 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268733    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.268899 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268775    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.268899 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268807    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.268899 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268845    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.268899 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268883    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.269148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.268914    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.269148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269004    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.269148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269035    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:09.269148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269053    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzm2m\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.269148 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269095    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket\") pod \"8752b1fd-af38-4aec-bf36-570e45ac65bd\" (UID: \"8752b1fd-af38-4aec-bf36-570e45ac65bd\") "
Sep 18 18:26:09.269456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269197    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:09.269456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269277    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:09.269456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269299    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:09.269628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269454    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:09.269628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269515    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.269628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269534    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.269628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269549    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.269628 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.269562    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8752b1fd-af38-4aec-bf36-570e45ac65bd-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.271125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.271099    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:09.271214 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.271158    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:09.271214 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.271166    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:26:09.271470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.271448    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m" (OuterVolumeSpecName: "kube-api-access-lzm2m") pod "8752b1fd-af38-4aec-bf36-570e45ac65bd" (UID: "8752b1fd-af38-4aec-bf36-570e45ac65bd"). InnerVolumeSpecName "kube-api-access-lzm2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:09.370501 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.370474    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzm2m\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-kube-api-access-lzm2m\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.370501 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.370498    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.370620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.370507    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.370620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.370516    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.370620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.370524    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8752b1fd-af38-4aec-bf36-570e45ac65bd-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:09.648389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.648365    2550 generic.go:356] "Generic (PLEG): container finished" podID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerID="2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825" exitCode=0
Sep 18 18:26:09.648509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.648412    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"
Sep 18 18:26:09.648509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.648451    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerDied","Data":"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"}
Sep 18 18:26:09.648585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.648512    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk" event={"ID":"8752b1fd-af38-4aec-bf36-570e45ac65bd","Type":"ContainerDied","Data":"50bfdf9d4f0cc99508b4c59ef96c8e495a421afd85152d3564586e1ea73e11d8"}
Sep 18 18:26:09.648585 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.648535    2550 scope.go:117] "RemoveContainer" containerID="2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"
Sep 18 18:26:09.660111 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.660096    2550 scope.go:117] "RemoveContainer" containerID="1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"
Sep 18 18:26:09.668191 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.668176    2550 scope.go:117] "RemoveContainer" containerID="2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"
Sep 18 18:26:09.668477 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:09.668461    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825\": container with ID starting with 2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825 not found: ID does not exist" containerID="2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"
Sep 18 18:26:09.668518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.668483    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825"} err="failed to get container status \"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825\": rpc error: code = NotFound desc = could not find container \"2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825\": container with ID starting with 2b454de8f71e5bf67a3094b1e99bb12fcdc8d79e4189c7349ed93c90422aa825 not found: ID does not exist"
Sep 18 18:26:09.668518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.668498    2550 scope.go:117] "RemoveContainer" containerID="1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"
Sep 18 18:26:09.668725 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:09.668703    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270\": container with ID starting with 1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270 not found: ID does not exist" containerID="1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"
Sep 18 18:26:09.668776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.668733    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270"} err="failed to get container status \"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270\": rpc error: code = NotFound desc = could not find container \"1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270\": container with ID starting with 1ab5d1473b6ab90101e0b9d6637179aa630f1497700c1e4e5b40e79b47f69270 not found: ID does not exist"
Sep 18 18:26:09.683347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.683325    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:26:09.686120 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:09.686102    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-e6108f-openshift-default-595bc8bb8-2c8fk"]
Sep 18 18:26:10.705591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:10.705549    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" path="/var/lib/kubelet/pods/8752b1fd-af38-4aec-bf36-570e45ac65bd/volumes"
Sep 18 18:26:11.277634 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:11.277606    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:26:20.339046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.339012    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:26:20.339555 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.339227    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" podUID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" containerName="payload-processing" containerID="cri-o://2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650" gracePeriod=30
Sep 18 18:26:20.519076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.519038    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"]
Sep 18 18:26:20.522265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.522240    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-29f0cb-b9sj4"]
Sep 18 18:26:20.586643 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.586620    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:26:20.663847 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.663754    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgmms\" (UniqueName: \"kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms\") pod \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") "
Sep 18 18:26:20.664044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.663868    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume\") pod \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\" (UID: \"85f19fe0-f794-447c-80e3-f7dda3cc28e9\") "
Sep 18 18:26:20.664266 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.664242    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85f19fe0-f794-447c-80e3-f7dda3cc28e9" (UID: "85f19fe0-f794-447c-80e3-f7dda3cc28e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:20.665765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.665738    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms" (OuterVolumeSpecName: "kube-api-access-dgmms") pod "85f19fe0-f794-447c-80e3-f7dda3cc28e9" (UID: "85f19fe0-f794-447c-80e3-f7dda3cc28e9"). InnerVolumeSpecName "kube-api-access-dgmms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:20.706240 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.706210    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e04c2a7-e60e-421d-8675-e50a59d600dd" path="/var/lib/kubelet/pods/8e04c2a7-e60e-421d-8675-e50a59d600dd/volumes"
Sep 18 18:26:20.709298 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.709273    2550 generic.go:356] "Generic (PLEG): container finished" podID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" containerID="2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650" exitCode=0
Sep 18 18:26:20.709440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.709301    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" event={"ID":"85f19fe0-f794-447c-80e3-f7dda3cc28e9","Type":"ContainerDied","Data":"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"}
Sep 18 18:26:20.709440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.709313    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"
Sep 18 18:26:20.709440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.709334    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f" event={"ID":"85f19fe0-f794-447c-80e3-f7dda3cc28e9","Type":"ContainerDied","Data":"a6ec8907051ba2c2e1e5bd1a6769ccffb9cbe670f8a2a712937b1984f5ec8e7c"}
Sep 18 18:26:20.709440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.709355    2550 scope.go:117] "RemoveContainer" containerID="2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"
Sep 18 18:26:20.719512 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.719495    2550 scope.go:117] "RemoveContainer" containerID="2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"
Sep 18 18:26:20.719768 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:20.719752    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650\": container with ID starting with 2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650 not found: ID does not exist" containerID="2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"
Sep 18 18:26:20.719822 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.719775    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650"} err="failed to get container status \"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650\": rpc error: code = NotFound desc = could not find container \"2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650\": container with ID starting with 2d47b5305933b06a882a685f63dfa66d0b41b9566aa7f0aefb403a9b48d88650 not found: ID does not exist"
Sep 18 18:26:20.729983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.729958    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:26:20.735666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.735643    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-29f0cb-6866d56597-cm92f"]
Sep 18 18:26:20.765133 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.765108    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgmms\" (UniqueName: \"kubernetes.io/projected/85f19fe0-f794-447c-80e3-f7dda3cc28e9-kube-api-access-dgmms\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:20.765133 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:20.765131    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f19fe0-f794-447c-80e3-f7dda3cc28e9-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:22.706063 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:22.706022    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" path="/var/lib/kubelet/pods/85f19fe0-f794-447c-80e3-f7dda3cc28e9/volumes"
Sep 18 18:26:27.236174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.236141    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:26:27.728255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728224    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"]
Sep 18 18:26:27.728599 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728586    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" containerName="payload-processing"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728600    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" containerName="payload-processing"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728607    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e04c2a7-e60e-421d-8675-e50a59d600dd" containerName="revoke-keys"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728613    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e04c2a7-e60e-421d-8675-e50a59d600dd" containerName="revoke-keys"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728624    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728629    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:27.728646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728647    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:27.728830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728653    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:27.728830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728708    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:27.728830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728716    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f19fe0-f794-447c-80e3-f7dda3cc28e9" containerName="payload-processing"
Sep 18 18:26:27.728830 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.728724    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e04c2a7-e60e-421d-8675-e50a59d600dd" containerName="revoke-keys"
Sep 18 18:26:27.732988 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.732960    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:27.735265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.735242    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:26:27.735406 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.735336    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:26:27.738510 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.738489    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"]
Sep 18 18:26:27.823260 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.823227    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt\" (UID: \"813e2fd4-b406-4919-ae9d-e3cf69655225\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:27.924304 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.924274    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt\" (UID: \"813e2fd4-b406-4919-ae9d-e3cf69655225\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:27.924879 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:27.924860    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt\" (UID: \"813e2fd4-b406-4919-ae9d-e3cf69655225\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:28.044777 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:28.044743    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:28.313794 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:28.313768    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"]
Sep 18 18:26:28.316034 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:26:28.316000    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813e2fd4_b406_4919_ae9d_e3cf69655225.slice/crio-1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe WatchSource:0}: Error finding container 1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe: Status 404 returned error can't find the container with id 1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe
Sep 18 18:26:28.753859 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:28.753818    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt" event={"ID":"813e2fd4-b406-4919-ae9d-e3cf69655225","Type":"ContainerStarted","Data":"92027a8857604dedf23e5ae98b231eddb22eedc6eebd1f985f62c64d497201fe"}
Sep 18 18:26:28.754074 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:28.753878    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt" event={"ID":"813e2fd4-b406-4919-ae9d-e3cf69655225","Type":"ContainerStarted","Data":"1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe"}
Sep 18 18:26:28.775987 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:28.775921    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt" podStartSLOduration=1.7759056819999999 podStartE2EDuration="1.775905682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:26:28.773261032 +0000 UTC m=+1560.459890485" watchObservedRunningTime="2026-09-18 18:26:28.775905682 +0000 UTC m=+1560.462535134"
Sep 18 18:26:29.760783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:29.760752    2550 generic.go:356] "Generic (PLEG): container finished" podID="813e2fd4-b406-4919-ae9d-e3cf69655225" containerID="92027a8857604dedf23e5ae98b231eddb22eedc6eebd1f985f62c64d497201fe" exitCode=0
Sep 18 18:26:29.761196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:29.760839    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt" event={"ID":"813e2fd4-b406-4919-ae9d-e3cf69655225","Type":"ContainerDied","Data":"92027a8857604dedf23e5ae98b231eddb22eedc6eebd1f985f62c64d497201fe"}
Sep 18 18:26:30.900998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:30.900975    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:30.949686 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:30.949663    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca\") pod \"813e2fd4-b406-4919-ae9d-e3cf69655225\" (UID: \"813e2fd4-b406-4919-ae9d-e3cf69655225\") "
Sep 18 18:26:30.950020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:30.949997    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "813e2fd4-b406-4919-ae9d-e3cf69655225" (UID: "813e2fd4-b406-4919-ae9d-e3cf69655225"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:30.958762 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:30.958741    2550 scope.go:117] "RemoveContainer" containerID="8599e27e1c12cd1ee09333bacc9fe68ea6a64de95c22fdb308329c404bf97f15"
Sep 18 18:26:31.050737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:31.050685    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/813e2fd4-b406-4919-ae9d-e3cf69655225-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:31.773213 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:31.773177    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"
Sep 18 18:26:31.773386 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:31.773210    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt" event={"ID":"813e2fd4-b406-4919-ae9d-e3cf69655225","Type":"ContainerDied","Data":"1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe"}
Sep 18 18:26:31.773386 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:31.773255    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ba98831d31a39335ad2a98d05c57ee14697b871bcf900e6b60853632b3a25fe"
Sep 18 18:26:36.543311 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.543276    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8897-26cb02"]
Sep 18 18:26:36.543679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.543630    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813e2fd4-b406-4919-ae9d-e3cf69655225" containerName="revoke-keys"
Sep 18 18:26:36.543679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.543643    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="813e2fd4-b406-4919-ae9d-e3cf69655225" containerName="revoke-keys"
Sep 18 18:26:36.543757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.543720    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="8752b1fd-af38-4aec-bf36-570e45ac65bd" containerName="istio-proxy"
Sep 18 18:26:36.543757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.543732    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="813e2fd4-b406-4919-ae9d-e3cf69655225" containerName="revoke-keys"
Sep 18 18:26:36.548394 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.548368    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:36.550422 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.550397    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-26cb02"]
Sep 18 18:26:36.550925 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.550907    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gxj55\""
Sep 18 18:26:36.597641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.597619    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwj55\" (UniqueName: \"kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55\") pod \"test-curl-8897-26cb02\" (UID: \"93151962-c239-457f-a7db-e57692aa0e0b\") " pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:36.698974 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.698933    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwj55\" (UniqueName: \"kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55\") pod \"test-curl-8897-26cb02\" (UID: \"93151962-c239-457f-a7db-e57692aa0e0b\") " pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:36.706780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.706750    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwj55\" (UniqueName: \"kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55\") pod \"test-curl-8897-26cb02\" (UID: \"93151962-c239-457f-a7db-e57692aa0e0b\") " pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:36.859041 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:36.858978    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:37.127990 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:37.127966    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-26cb02"]
Sep 18 18:26:37.130007 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:26:37.129976    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93151962_c239_457f_a7db_e57692aa0e0b.slice/crio-58e6e8706748d511e4064ef2f9766563aae8d36be10aef052011a3d7d5947618 WatchSource:0}: Error finding container 58e6e8706748d511e4064ef2f9766563aae8d36be10aef052011a3d7d5947618: Status 404 returned error can't find the container with id 58e6e8706748d511e4064ef2f9766563aae8d36be10aef052011a3d7d5947618
Sep 18 18:26:37.805437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:37.805398    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-26cb02" event={"ID":"93151962-c239-457f-a7db-e57692aa0e0b","Type":"ContainerStarted","Data":"09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639"}
Sep 18 18:26:37.805799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:37.805447    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-26cb02" event={"ID":"93151962-c239-457f-a7db-e57692aa0e0b","Type":"ContainerStarted","Data":"58e6e8706748d511e4064ef2f9766563aae8d36be10aef052011a3d7d5947618"}
Sep 18 18:26:37.819096 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:37.819050    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8897-26cb02" podStartSLOduration=1.819033533 podStartE2EDuration="1.819033533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:26:37.818184318 +0000 UTC m=+1569.504813792" watchObservedRunningTime="2026-09-18 18:26:37.819033533 +0000 UTC m=+1569.505662993"
Sep 18 18:26:38.437277 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.437239    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"]
Sep 18 18:26:38.440417 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.440391    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8897-26cb02"]
Sep 18 18:26:38.443314 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.443288    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-f5e6d4-ggxrt"]
Sep 18 18:26:38.445403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.445377    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8897-26cb02"]
Sep 18 18:26:38.571130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.571103    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8897-123826"]
Sep 18 18:26:38.571456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.571444    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93151962-c239-457f-a7db-e57692aa0e0b" containerName="test-curl-8897-26cb02"
Sep 18 18:26:38.571505 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.571458    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="93151962-c239-457f-a7db-e57692aa0e0b" containerName="test-curl-8897-26cb02"
Sep 18 18:26:38.571551 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.571541    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="93151962-c239-457f-a7db-e57692aa0e0b" containerName="test-curl-8897-26cb02"
Sep 18 18:26:38.574492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.574475    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:38.582497 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.582473    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-123826"]
Sep 18 18:26:38.706043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.706017    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813e2fd4-b406-4919-ae9d-e3cf69655225" path="/var/lib/kubelet/pods/813e2fd4-b406-4919-ae9d-e3cf69655225/volumes"
Sep 18 18:26:38.715354 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.715332    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slbmm\" (UniqueName: \"kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm\") pod \"test-curl-8897-123826\" (UID: \"b615072f-8e1f-41ca-9e8e-4c43bdce2e02\") " pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:38.810704 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:38.810685    2550 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8897-26cb02" container="test-curl-8897-26cb02"
Sep 18 18:26:38.815911 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.815893    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbmm\" (UniqueName: \"kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm\") pod \"test-curl-8897-123826\" (UID: \"b615072f-8e1f-41ca-9e8e-4c43bdce2e02\") " pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:38.823320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.823272    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slbmm\" (UniqueName: \"kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm\") pod \"test-curl-8897-123826\" (UID: \"b615072f-8e1f-41ca-9e8e-4c43bdce2e02\") " pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:38.883677 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:38.883659    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:39.164983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.164956    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8897-123826"]
Sep 18 18:26:39.166983 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:26:39.166924    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb615072f_8e1f_41ca_9e8e_4c43bdce2e02.slice/crio-efbca41fd69347867d3d3e8e83dba2a69e9c4176d9da051373313bbbbe19c504 WatchSource:0}: Error finding container efbca41fd69347867d3d3e8e83dba2a69e9c4176d9da051373313bbbbe19c504: Status 404 returned error can't find the container with id efbca41fd69347867d3d3e8e83dba2a69e9c4176d9da051373313bbbbe19c504
Sep 18 18:26:39.679129 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.679104    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:26:39.817026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.816994    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-123826" event={"ID":"b615072f-8e1f-41ca-9e8e-4c43bdce2e02","Type":"ContainerStarted","Data":"a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd"}
Sep 18 18:26:39.817479 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.817032    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8897-123826" event={"ID":"b615072f-8e1f-41ca-9e8e-4c43bdce2e02","Type":"ContainerStarted","Data":"efbca41fd69347867d3d3e8e83dba2a69e9c4176d9da051373313bbbbe19c504"}
Sep 18 18:26:39.817479 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.817197    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8897-26cb02" podUID="93151962-c239-457f-a7db-e57692aa0e0b" containerName="test-curl-8897-26cb02" containerID="cri-o://09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639" gracePeriod=2
Sep 18 18:26:39.832288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:39.832229    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-8897-123826" podStartSLOduration=1.8322166549999999 podStartE2EDuration="1.832216655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:26:39.830807998 +0000 UTC m=+1571.517437449" watchObservedRunningTime="2026-09-18 18:26:39.832216655 +0000 UTC m=+1571.518846106"
Sep 18 18:26:40.430969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:40.430923    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8897-123826"]
Sep 18 18:26:40.433317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:40.433291    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8897-123826"]
Sep 18 18:26:40.821451 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:40.821423    2550 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-8897-123826" container="test-curl-8897-123826"
Sep 18 18:26:41.827254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:41.827220    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-8897-123826" podUID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" containerName="test-curl-8897-123826" containerID="cri-o://a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd" gracePeriod=2
Sep 18 18:26:41.988357 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:41.988333    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:42.039684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.039662    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwj55\" (UniqueName: \"kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55\") pod \"93151962-c239-457f-a7db-e57692aa0e0b\" (UID: \"93151962-c239-457f-a7db-e57692aa0e0b\") "
Sep 18 18:26:42.041579 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.041544    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55" (OuterVolumeSpecName: "kube-api-access-vwj55") pod "93151962-c239-457f-a7db-e57692aa0e0b" (UID: "93151962-c239-457f-a7db-e57692aa0e0b"). InnerVolumeSpecName "kube-api-access-vwj55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:42.140697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.140645    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwj55\" (UniqueName: \"kubernetes.io/projected/93151962-c239-457f-a7db-e57692aa0e0b-kube-api-access-vwj55\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:42.515403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.515370    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:26:42.706054 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.706026    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93151962-c239-457f-a7db-e57692aa0e0b" path="/var/lib/kubelet/pods/93151962-c239-457f-a7db-e57692aa0e0b/volumes"
Sep 18 18:26:42.833572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.833504    2550 generic.go:356] "Generic (PLEG): container finished" podID="93151962-c239-457f-a7db-e57692aa0e0b" containerID="09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639" exitCode=137
Sep 18 18:26:42.833572 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.833539    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-26cb02"
Sep 18 18:26:42.833933 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.833603    2550 scope.go:117] "RemoveContainer" containerID="09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639"
Sep 18 18:26:42.841912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.841897    2550 scope.go:117] "RemoveContainer" containerID="09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639"
Sep 18 18:26:42.842195 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:42.842174    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639\": container with ID starting with 09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639 not found: ID does not exist" containerID="09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639"
Sep 18 18:26:42.842309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:42.842202    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639"} err="failed to get container status \"09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639\": rpc error: code = NotFound desc = could not find container \"09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639\": container with ID starting with 09e96f62c3b4d8bda395eccd07a8d2c86cef6592a0da0b2cb6d2bf6129f6d639 not found: ID does not exist"
Sep 18 18:26:43.596064 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:43.596027    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:26:44.013173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.013147    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:44.154694 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.154631    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slbmm\" (UniqueName: \"kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm\") pod \"b615072f-8e1f-41ca-9e8e-4c43bdce2e02\" (UID: \"b615072f-8e1f-41ca-9e8e-4c43bdce2e02\") "
Sep 18 18:26:44.156602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.156577    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm" (OuterVolumeSpecName: "kube-api-access-slbmm") pod "b615072f-8e1f-41ca-9e8e-4c43bdce2e02" (UID: "b615072f-8e1f-41ca-9e8e-4c43bdce2e02"). InnerVolumeSpecName "kube-api-access-slbmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:44.255996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.255971    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slbmm\" (UniqueName: \"kubernetes.io/projected/b615072f-8e1f-41ca-9e8e-4c43bdce2e02-kube-api-access-slbmm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:44.706409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.706381    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" path="/var/lib/kubelet/pods/b615072f-8e1f-41ca-9e8e-4c43bdce2e02/volumes"
Sep 18 18:26:44.847255 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.847224    2550 generic.go:356] "Generic (PLEG): container finished" podID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" containerID="a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd" exitCode=137
Sep 18 18:26:44.847376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.847265    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8897-123826"
Sep 18 18:26:44.847376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.847310    2550 scope.go:117] "RemoveContainer" containerID="a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd"
Sep 18 18:26:44.855676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.855660    2550 scope.go:117] "RemoveContainer" containerID="a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd"
Sep 18 18:26:44.855947 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:44.855921    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd\": container with ID starting with a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd not found: ID does not exist" containerID="a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd"
Sep 18 18:26:44.855994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:44.855956    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd"} err="failed to get container status \"a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd\": rpc error: code = NotFound desc = could not find container \"a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd\": container with ID starting with a90b7a77cbe01eb210881d06ec099647db45666373a98ab7fdb214dc19f0bddd not found: ID does not exist"
Sep 18 18:26:45.193438 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.193415    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"]
Sep 18 18:26:45.193812 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.193799    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" containerName="test-curl-8897-123826"
Sep 18 18:26:45.193869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.193814    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" containerName="test-curl-8897-123826"
Sep 18 18:26:45.193903 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.193875    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b615072f-8e1f-41ca-9e8e-4c43bdce2e02" containerName="test-curl-8897-123826"
Sep 18 18:26:45.196706 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.196691    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:45.199289 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.199270    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:26:45.199289 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.199283    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:26:45.204743 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.204696    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"]
Sep 18 18:26:45.265435 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.265411    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf\" (UID: \"6fac243d-04f6-4de1-8813-07a2b145b83e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:45.366529 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.366502    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf\" (UID: \"6fac243d-04f6-4de1-8813-07a2b145b83e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:45.367099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.367081    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf\" (UID: \"6fac243d-04f6-4de1-8813-07a2b145b83e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:45.508261 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.508242    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:45.783887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.783864    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"]
Sep 18 18:26:45.786057 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:26:45.786028    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fac243d_04f6_4de1_8813_07a2b145b83e.slice/crio-b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61 WatchSource:0}: Error finding container b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61: Status 404 returned error can't find the container with id b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61
Sep 18 18:26:45.853861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:45.853836    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf" event={"ID":"6fac243d-04f6-4de1-8813-07a2b145b83e","Type":"ContainerStarted","Data":"b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61"}
Sep 18 18:26:46.861556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:46.861527    2550 generic.go:356] "Generic (PLEG): container finished" podID="6fac243d-04f6-4de1-8813-07a2b145b83e" containerID="6eee4cbe605d7d078c37eb4697dddeb6ac6920b42ed752822ca724107cc93cd8" exitCode=0
Sep 18 18:26:46.861879 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:46.861611    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf" event={"ID":"6fac243d-04f6-4de1-8813-07a2b145b83e","Type":"ContainerDied","Data":"6eee4cbe605d7d078c37eb4697dddeb6ac6920b42ed752822ca724107cc93cd8"}
Sep 18 18:26:47.997461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:47.997441    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:48.091167 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.091143    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca\") pod \"6fac243d-04f6-4de1-8813-07a2b145b83e\" (UID: \"6fac243d-04f6-4de1-8813-07a2b145b83e\") "
Sep 18 18:26:48.091469 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.091447    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fac243d-04f6-4de1-8813-07a2b145b83e" (UID: "6fac243d-04f6-4de1-8813-07a2b145b83e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:48.192445 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.192385    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fac243d-04f6-4de1-8813-07a2b145b83e-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:48.872994 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.872964    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf" event={"ID":"6fac243d-04f6-4de1-8813-07a2b145b83e","Type":"ContainerDied","Data":"b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61"}
Sep 18 18:26:48.873127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.872999    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"
Sep 18 18:26:48.873127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:48.873018    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0c1f65e19076f6bebed257119a7aabbf3ea01d133850d494257f290b32d4e61"
Sep 18 18:26:53.489027 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:53.488990    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:26:55.773773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.773739    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:26:55.774250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.773975    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" podUID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" containerName="payload-processing" containerID="cri-o://a26a26f9a6215fe65f625901dc6da18056c8f82c0d4e8effe22c69c28629305c" gracePeriod=30
Sep 18 18:26:55.779755 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.779707    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:26:55.780114 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.780076    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" podUID="bb02a275-aba3-46ff-bdea-480102ada8b0" containerName="payload-pre-processing" containerID="cri-o://7b0ca4685c6e19ed3281d4881cefbb480a6fbf1c50bf062511c978f55b66b301" gracePeriod=30
Sep 18 18:26:55.911281 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.911243    2550 generic.go:356] "Generic (PLEG): container finished" podID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" containerID="a26a26f9a6215fe65f625901dc6da18056c8f82c0d4e8effe22c69c28629305c" exitCode=0
Sep 18 18:26:55.911486 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.911320    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" event={"ID":"bf8fce21-6f3f-4cd1-a74d-fefe4109be87","Type":"ContainerDied","Data":"a26a26f9a6215fe65f625901dc6da18056c8f82c0d4e8effe22c69c28629305c"}
Sep 18 18:26:55.913614 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.913589    2550 generic.go:356] "Generic (PLEG): container finished" podID="bb02a275-aba3-46ff-bdea-480102ada8b0" containerID="7b0ca4685c6e19ed3281d4881cefbb480a6fbf1c50bf062511c978f55b66b301" exitCode=0
Sep 18 18:26:55.913742 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.913658    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" event={"ID":"bb02a275-aba3-46ff-bdea-480102ada8b0","Type":"ContainerDied","Data":"7b0ca4685c6e19ed3281d4881cefbb480a6fbf1c50bf062511c978f55b66b301"}
Sep 18 18:26:55.965050 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.965020    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"]
Sep 18 18:26:55.969059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:55.969036    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-1f7000-mqthf"]
Sep 18 18:26:56.056869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.056844    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:26:56.059651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.059628    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:26:56.157436 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157399    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-428th\" (UniqueName: \"kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th\") pod \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") "
Sep 18 18:26:56.157646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157491    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume\") pod \"bb02a275-aba3-46ff-bdea-480102ada8b0\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") "
Sep 18 18:26:56.157646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157520    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume\") pod \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\" (UID: \"bf8fce21-6f3f-4cd1-a74d-fefe4109be87\") "
Sep 18 18:26:56.157646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157537    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxxq4\" (UniqueName: \"kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4\") pod \"bb02a275-aba3-46ff-bdea-480102ada8b0\" (UID: \"bb02a275-aba3-46ff-bdea-480102ada8b0\") "
Sep 18 18:26:56.157857 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157828    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb02a275-aba3-46ff-bdea-480102ada8b0" (UID: "bb02a275-aba3-46ff-bdea-480102ada8b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:56.157972 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.157948    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf8fce21-6f3f-4cd1-a74d-fefe4109be87" (UID: "bf8fce21-6f3f-4cd1-a74d-fefe4109be87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:26:56.159711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.159687    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4" (OuterVolumeSpecName: "kube-api-access-fxxq4") pod "bb02a275-aba3-46ff-bdea-480102ada8b0" (UID: "bb02a275-aba3-46ff-bdea-480102ada8b0"). InnerVolumeSpecName "kube-api-access-fxxq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:56.159711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.159699    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th" (OuterVolumeSpecName: "kube-api-access-428th") pod "bf8fce21-6f3f-4cd1-a74d-fefe4109be87" (UID: "bf8fce21-6f3f-4cd1-a74d-fefe4109be87"). InnerVolumeSpecName "kube-api-access-428th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:56.258689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.258648    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.258689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.258686    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxxq4\" (UniqueName: \"kubernetes.io/projected/bb02a275-aba3-46ff-bdea-480102ada8b0-kube-api-access-fxxq4\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.258689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.258696    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-428th\" (UniqueName: \"kubernetes.io/projected/bf8fce21-6f3f-4cd1-a74d-fefe4109be87-kube-api-access-428th\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.258689 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.258705    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb02a275-aba3-46ff-bdea-480102ada8b0-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:56.707358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.707319    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fac243d-04f6-4de1-8813-07a2b145b83e" path="/var/lib/kubelet/pods/6fac243d-04f6-4de1-8813-07a2b145b83e/volumes"
Sep 18 18:26:56.920719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.920683    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"
Sep 18 18:26:56.920719 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.920697    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj" event={"ID":"bf8fce21-6f3f-4cd1-a74d-fefe4109be87","Type":"ContainerDied","Data":"e3b12b7458d42edb62c54cb3d7c1a246f7a1acbe75b9ea6f4fac323623bf0284"}
Sep 18 18:26:56.921261 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.920758    2550 scope.go:117] "RemoveContainer" containerID="a26a26f9a6215fe65f625901dc6da18056c8f82c0d4e8effe22c69c28629305c"
Sep 18 18:26:56.922198 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.922143    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"
Sep 18 18:26:56.922198 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.922146    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp" event={"ID":"bb02a275-aba3-46ff-bdea-480102ada8b0","Type":"ContainerDied","Data":"a253c2b1e80a5b236e5b06cd3fec3651dece5cfd9786d8e47b1521ed495b51d0"}
Sep 18 18:26:56.929912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.929894    2550 scope.go:117] "RemoveContainer" containerID="7b0ca4685c6e19ed3281d4881cefbb480a6fbf1c50bf062511c978f55b66b301"
Sep 18 18:26:56.939771 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.939742    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:26:56.941784 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.941763    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-1f7000-b4c595565-6mcrj"]
Sep 18 18:26:56.951909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.951887    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:26:56.955590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:56.955566    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-1f7000-7f68dfc46-p2drp"]
Sep 18 18:26:57.534430 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.534381    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" podUID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" containerName="main" containerID="cri-o://fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b" gracePeriod=45
Sep 18 18:26:57.783829 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.783800    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:26:57.875659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875573    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wprq\" (UniqueName: \"kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.875659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875639    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.875887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875678    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.875887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875747    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.875887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875814    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.875887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875842    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs\") pod \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\" (UID: \"b357868a-6f1f-4c0d-b73e-85fa666f8a2d\") "
Sep 18 18:26:57.876107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.875979    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache" (OuterVolumeSpecName: "model-cache") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:57.876107 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.876018    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:57.876187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.876092    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home" (OuterVolumeSpecName: "home") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:57.876251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.876234    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:57.876300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.876256    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:57.876300 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.876268    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:57.878019 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.877995    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:57.878267 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.878243    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm" (OuterVolumeSpecName: "dshm") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:57.878367 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.878279    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq" (OuterVolumeSpecName: "kube-api-access-8wprq") pod "b357868a-6f1f-4c0d-b73e-85fa666f8a2d" (UID: "b357868a-6f1f-4c0d-b73e-85fa666f8a2d"). InnerVolumeSpecName "kube-api-access-8wprq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:57.930810 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.930772    2550 generic.go:356] "Generic (PLEG): container finished" podID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" containerID="fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b" exitCode=0
Sep 18 18:26:57.930810 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.930813    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"
Sep 18 18:26:57.931358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.930864    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" event={"ID":"b357868a-6f1f-4c0d-b73e-85fa666f8a2d","Type":"ContainerDied","Data":"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"}
Sep 18 18:26:57.931358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.930905    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx" event={"ID":"b357868a-6f1f-4c0d-b73e-85fa666f8a2d","Type":"ContainerDied","Data":"8d16cab4eda49f6ed138a229068b52e45701ceb49f8e8ace8e4f10fbabc1e152"}
Sep 18 18:26:57.931358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.930925    2550 scope.go:117] "RemoveContainer" containerID="fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"
Sep 18 18:26:57.940683 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.940662    2550 scope.go:117] "RemoveContainer" containerID="fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"
Sep 18 18:26:57.940980 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:57.940961    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b\": container with ID starting with fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b not found: ID does not exist" containerID="fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"
Sep 18 18:26:57.941030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.940989    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b"} err="failed to get container status \"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b\": rpc error: code = NotFound desc = could not find container \"fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b\": container with ID starting with fb11436dca5b8f2dcaef75b5df79d52c2e3999583fd8b9b42c27ad32856d868b not found: ID does not exist"
Sep 18 18:26:57.951812 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.951785    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:26:57.954526 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.954504    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-011541/sub-test-model-011541-kserve-84dbf65459-vs5jx"]
Sep 18 18:26:57.976976 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.976931    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:57.976976 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.976978    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wprq\" (UniqueName: \"kubernetes.io/projected/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-kube-api-access-8wprq\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:57.977180 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:57.976987    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b357868a-6f1f-4c0d-b73e-85fa666f8a2d-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:58.617002 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.616960    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" podUID="e2808838-f633-44ca-80d7-f474857033c9" containerName="main" containerID="cri-o://ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae" gracePeriod=45
Sep 18 18:26:58.706851 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.706822    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" path="/var/lib/kubelet/pods/b357868a-6f1f-4c0d-b73e-85fa666f8a2d/volumes"
Sep 18 18:26:58.707216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.707201    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb02a275-aba3-46ff-bdea-480102ada8b0" path="/var/lib/kubelet/pods/bb02a275-aba3-46ff-bdea-480102ada8b0/volumes"
Sep 18 18:26:58.707519 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.707508    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" path="/var/lib/kubelet/pods/bf8fce21-6f3f-4cd1-a74d-fefe4109be87/volumes"
Sep 18 18:26:58.859278 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.859252    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:26:58.938620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.938521    2550 generic.go:356] "Generic (PLEG): container finished" podID="e2808838-f633-44ca-80d7-f474857033c9" containerID="ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae" exitCode=0
Sep 18 18:26:58.938620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.938571    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"
Sep 18 18:26:58.939159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.938570    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" event={"ID":"e2808838-f633-44ca-80d7-f474857033c9","Type":"ContainerDied","Data":"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"}
Sep 18 18:26:58.939159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.938672    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r" event={"ID":"e2808838-f633-44ca-80d7-f474857033c9","Type":"ContainerDied","Data":"9999cd0d23a6e1c6c601ecb4781bfb70722df315ed6ac64c569ea3773f350010"}
Sep 18 18:26:58.939159 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.938703    2550 scope.go:117] "RemoveContainer" containerID="ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"
Sep 18 18:26:58.948509 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.948491    2550 scope.go:117] "RemoveContainer" containerID="ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"
Sep 18 18:26:58.948817 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:26:58.948794    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae\": container with ID starting with ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae not found: ID does not exist" containerID="ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"
Sep 18 18:26:58.948868 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.948831    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae"} err="failed to get container status \"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae\": rpc error: code = NotFound desc = could not find container \"ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae\": container with ID starting with ac29c8875dbe4aa7c695ef4cb43240f26fee9750f5e2c86a40257581f8c4f5ae not found: ID does not exist"
Sep 18 18:26:58.985783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985751    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.985970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985833    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.985970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985855    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.985970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985887    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f64th\" (UniqueName: \"kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.985970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985924    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.985970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.985967    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs\") pod \"e2808838-f633-44ca-80d7-f474857033c9\" (UID: \"e2808838-f633-44ca-80d7-f474857033c9\") "
Sep 18 18:26:58.986317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.986053    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home" (OuterVolumeSpecName: "home") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:58.986317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.986157    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:58.986317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.986201    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:58.986317 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.986236    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:58.988060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.988037    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:26:58.988060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.988052    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm" (OuterVolumeSpecName: "dshm") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:26:58.988060 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:58.988042    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th" (OuterVolumeSpecName: "kube-api-access-f64th") pod "e2808838-f633-44ca-80d7-f474857033c9" (UID: "e2808838-f633-44ca-80d7-f474857033c9"). InnerVolumeSpecName "kube-api-access-f64th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:26:59.086889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.086856    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:59.086889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.086886    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2808838-f633-44ca-80d7-f474857033c9-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:59.086889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.086895    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:59.087130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.086903    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2808838-f633-44ca-80d7-f474857033c9-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:59.087130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.086915    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f64th\" (UniqueName: \"kubernetes.io/projected/e2808838-f633-44ca-80d7-f474857033c9-kube-api-access-f64th\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:26:59.263343 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.261172    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:26:59.267690 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:26:59.267658    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/sub-test-model-99602c-kserve-7fc9c48457-zfl2r"]
Sep 18 18:27:00.706153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:00.706121    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2808838-f633-44ca-80d7-f474857033c9" path="/var/lib/kubelet/pods/e2808838-f633-44ca-80d7-f474857033c9/volumes"
Sep 18 18:27:03.284611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:03.284579    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:27:17.786920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.786837    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787213    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" containerName="payload-processing"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787226    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" containerName="payload-processing"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787238    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2808838-f633-44ca-80d7-f474857033c9" containerName="main"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787247    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2808838-f633-44ca-80d7-f474857033c9" containerName="main"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787257    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb02a275-aba3-46ff-bdea-480102ada8b0" containerName="payload-pre-processing"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787263    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb02a275-aba3-46ff-bdea-480102ada8b0" containerName="payload-pre-processing"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787270    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" containerName="main"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787276    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" containerName="main"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787285    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fac243d-04f6-4de1-8813-07a2b145b83e" containerName="revoke-keys"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787291    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fac243d-04f6-4de1-8813-07a2b145b83e" containerName="revoke-keys"
Sep 18 18:27:17.787362 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787364    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fac243d-04f6-4de1-8813-07a2b145b83e" containerName="revoke-keys"
Sep 18 18:27:17.787702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787371    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b357868a-6f1f-4c0d-b73e-85fa666f8a2d" containerName="main"
Sep 18 18:27:17.787702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787378    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb02a275-aba3-46ff-bdea-480102ada8b0" containerName="payload-pre-processing"
Sep 18 18:27:17.787702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787385    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2808838-f633-44ca-80d7-f474857033c9" containerName="main"
Sep 18 18:27:17.787702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.787390    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf8fce21-6f3f-4cd1-a74d-fefe4109be87" containerName="payload-processing"
Sep 18 18:27:17.791668 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.791650    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.794797 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.794773    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"default-dockercfg-r7zmk\""
Sep 18 18:27:17.794916 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.794826    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"openshift-service-ca.crt\""
Sep 18 18:27:17.794916 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.794857    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"kube-root-ca.crt\""
Sep 18 18:27:17.794916 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.794886    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-99602c\"/\"rate-test-model-99602c-kserve-self-signed-certs\""
Sep 18 18:27:17.797783 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.797762    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:27:17.824587 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824566    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.824682 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824610    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.824682 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824633    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.824682 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824670    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.824791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824704    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wspdq\" (UniqueName: \"kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.824791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.824741    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925808 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925785    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925817    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925842    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925859    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925880    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.925926 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.925923    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wspdq\" (UniqueName: \"kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.926309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.926286    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.926376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.926288    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.926376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.926332    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.928155 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.928134    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.928334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.928318    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:17.936571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:17.936551    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wspdq\" (UniqueName: \"kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq\") pod \"rate-test-model-99602c-kserve-ffcd6b495-k9scg\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") " pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:18.102531 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:18.102473    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:18.373841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:18.373796    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:27:18.376165 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:27:18.376140    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcdd33fca_b1d6_4478_8605_433c7941229a.slice/crio-39b47952214bd30d39c1845781c24a5cb205e5a76e3dc7be6ab8971b365f4ea1 WatchSource:0}: Error finding container 39b47952214bd30d39c1845781c24a5cb205e5a76e3dc7be6ab8971b365f4ea1: Status 404 returned error can't find the container with id 39b47952214bd30d39c1845781c24a5cb205e5a76e3dc7be6ab8971b365f4ea1
Sep 18 18:27:19.056174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:19.056134    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" event={"ID":"cdd33fca-b1d6-4478-8605-433c7941229a","Type":"ContainerStarted","Data":"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"}
Sep 18 18:27:19.056174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:19.056182    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" event={"ID":"cdd33fca-b1d6-4478-8605-433c7941229a","Type":"ContainerStarted","Data":"39b47952214bd30d39c1845781c24a5cb205e5a76e3dc7be6ab8971b365f4ea1"}
Sep 18 18:27:19.075780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:19.075726    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" podStartSLOduration=2.075710793 podStartE2EDuration="2.075710793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:27:19.074232226 +0000 UTC m=+1610.760861684" watchObservedRunningTime="2026-09-18 18:27:19.075710793 +0000 UTC m=+1610.762340245"
Sep 18 18:27:21.978007 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:21.977973    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:27:28.102909 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:28.102875    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:28.103514 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:28.102964    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:28.116051 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:28.116022    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:29.122164 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:29.122131    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:27:36.574173 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:36.574128    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:27:46.175355 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:46.175319    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:27:53.783807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:53.783775    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:27:56.884289 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:27:56.884249    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:28:00.065215 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.065184    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:28:00.068630 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.068614    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.070923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.070900    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-15e8cc\""
Sep 18 18:28:00.071078 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.070951    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-15e8cc-dockercfg-wx9pr\""
Sep 18 18:28:00.078020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.077996    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:28:00.111709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.111685    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:28:00.114947 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.114924    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.128450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.128430    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:28:00.189256 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.189215    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.189399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.189272    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bjbc\" (UniqueName: \"kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.290072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.290046    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.290178 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.290090    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfz6\" (UniqueName: \"kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.290224 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.290184    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.290262 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.290230    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bjbc\" (UniqueName: \"kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.290958 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.290915    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.297829 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.297812    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bjbc\" (UniqueName: \"kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc\") pod \"payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.378766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.378710    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:00.391174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.391156    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfz6\" (UniqueName: \"kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.391237 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.391217    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.391791 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.391773    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.398890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.398867    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfz6\" (UniqueName: \"kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6\") pod \"payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.424903 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.424873    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:00.648608 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.648581    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:28:00.650629 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:28:00.650601    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf72e136_c898_4cd2_a2d8_d0e34e360085.slice/crio-2c798fd357f35850971b6c948e8d22b9de61c4bd9595ed5c9ae024ce3c889890 WatchSource:0}: Error finding container 2c798fd357f35850971b6c948e8d22b9de61c4bd9595ed5c9ae024ce3c889890: Status 404 returned error can't find the container with id 2c798fd357f35850971b6c948e8d22b9de61c4bd9595ed5c9ae024ce3c889890
Sep 18 18:28:00.706139 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:28:00.706117    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a696c9f_849d_436a_bb32_b52faf3087cf.slice/crio-cf3ac64684ae2c92e91dd3e113b48127c03237372c7f476cc32425623a81e02a WatchSource:0}: Error finding container cf3ac64684ae2c92e91dd3e113b48127c03237372c7f476cc32425623a81e02a: Status 404 returned error can't find the container with id cf3ac64684ae2c92e91dd3e113b48127c03237372c7f476cc32425623a81e02a
Sep 18 18:28:00.708208 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:00.708182    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:28:01.280576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.280541    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" event={"ID":"df72e136-c898-4cd2-a2d8-d0e34e360085","Type":"ContainerStarted","Data":"a470758cf56769c904b68629ceb2827bb837796364f69311e1ca62c33d19a285"}
Sep 18 18:28:01.281069 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.280610    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:01.281069 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.280634    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" event={"ID":"df72e136-c898-4cd2-a2d8-d0e34e360085","Type":"ContainerStarted","Data":"2c798fd357f35850971b6c948e8d22b9de61c4bd9595ed5c9ae024ce3c889890"}
Sep 18 18:28:01.282355 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.282325    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" event={"ID":"9a696c9f-849d-436a-bb32-b52faf3087cf","Type":"ContainerStarted","Data":"c788679df34f945313aa4e3f2c2156bff00222e4a4b8360c23a48f6b901e645a"}
Sep 18 18:28:01.282502 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.282374    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" event={"ID":"9a696c9f-849d-436a-bb32-b52faf3087cf","Type":"ContainerStarted","Data":"cf3ac64684ae2c92e91dd3e113b48127c03237372c7f476cc32425623a81e02a"}
Sep 18 18:28:01.282502 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.282406    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:01.295206 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.295165    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" podStartSLOduration=1.295148337 podStartE2EDuration="1.295148337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:28:01.294823733 +0000 UTC m=+1652.981453201" watchObservedRunningTime="2026-09-18 18:28:01.295148337 +0000 UTC m=+1652.981777789"
Sep 18 18:28:01.310672 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:01.310626    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" podStartSLOduration=1.310612622 podStartE2EDuration="1.310612622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:28:01.308486802 +0000 UTC m=+1652.995150211" watchObservedRunningTime="2026-09-18 18:28:01.310612622 +0000 UTC m=+1652.997242073"
Sep 18 18:28:08.802611 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:08.802574    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" podUID="cdd33fca-b1d6-4478-8605-433c7941229a" containerName="main" containerID="cri-o://65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e" gracePeriod=45
Sep 18 18:28:09.053314 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.053259    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:28:09.060655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060633    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.060778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060677    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.060778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060703    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wspdq\" (UniqueName: \"kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.060778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060722    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.060778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060742    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.060778 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060770    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache\") pod \"cdd33fca-b1d6-4478-8605-433c7941229a\" (UID: \"cdd33fca-b1d6-4478-8605-433c7941229a\") "
Sep 18 18:28:09.061073 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.060892    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:28:09.061164 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.061144    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.061216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.061147    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home" (OuterVolumeSpecName: "home") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:28:09.061216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.061206    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache" (OuterVolumeSpecName: "model-cache") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:28:09.063250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.063224    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm" (OuterVolumeSpecName: "dshm") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:28:09.063358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.063327    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:28:09.063358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.063334    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq" (OuterVolumeSpecName: "kube-api-access-wspdq") pod "cdd33fca-b1d6-4478-8605-433c7941229a" (UID: "cdd33fca-b1d6-4478-8605-433c7941229a"). InnerVolumeSpecName "kube-api-access-wspdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:28:09.161602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.161572    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.161602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.161601    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wspdq\" (UniqueName: \"kubernetes.io/projected/cdd33fca-b1d6-4478-8605-433c7941229a-kube-api-access-wspdq\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.161733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.161612    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cdd33fca-b1d6-4478-8605-433c7941229a-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.161733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.161621    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.161733 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.161628    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cdd33fca-b1d6-4478-8605-433c7941229a-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:28:09.329434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.329376    2550 generic.go:356] "Generic (PLEG): container finished" podID="cdd33fca-b1d6-4478-8605-433c7941229a" containerID="65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e" exitCode=0
Sep 18 18:28:09.329434 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.329415    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" event={"ID":"cdd33fca-b1d6-4478-8605-433c7941229a","Type":"ContainerDied","Data":"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"}
Sep 18 18:28:09.329562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.329448    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg" event={"ID":"cdd33fca-b1d6-4478-8605-433c7941229a","Type":"ContainerDied","Data":"39b47952214bd30d39c1845781c24a5cb205e5a76e3dc7be6ab8971b365f4ea1"}
Sep 18 18:28:09.329562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.329464    2550 scope.go:117] "RemoveContainer" containerID="65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"
Sep 18 18:28:09.329562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.329422    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"
Sep 18 18:28:09.338250 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.338232    2550 scope.go:117] "RemoveContainer" containerID="65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"
Sep 18 18:28:09.338514 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:28:09.338498    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e\": container with ID starting with 65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e not found: ID does not exist" containerID="65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"
Sep 18 18:28:09.338560 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.338522    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e"} err="failed to get container status \"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e\": rpc error: code = NotFound desc = could not find container \"65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e\": container with ID starting with 65d36c71beca0413fb4e4b53e17b8aa9b1450b16f3a4814d3aabfcb7d50ea28e not found: ID does not exist"
Sep 18 18:28:09.349444 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.349417    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:28:09.353837 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:09.353813    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-99602c/rate-test-model-99602c-kserve-ffcd6b495-k9scg"]
Sep 18 18:28:10.706216 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:10.706185    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdd33fca-b1d6-4478-8605-433c7941229a" path="/var/lib/kubelet/pods/cdd33fca-b1d6-4478-8605-433c7941229a/volumes"
Sep 18 18:28:10.879357 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:10.879319    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:28:12.290124 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:12.290093    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:28:12.290534 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:12.290258    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:28:30.706429 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.706397    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:28:30.706930 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.706908    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdd33fca-b1d6-4478-8605-433c7941229a" containerName="main"
Sep 18 18:28:30.707032 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.706932    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdd33fca-b1d6-4478-8605-433c7941229a" containerName="main"
Sep 18 18:28:30.707104 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.707041    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdd33fca-b1d6-4478-8605-433c7941229a" containerName="main"
Sep 18 18:28:30.709754 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.709734    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.712755 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.712729    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-46c63c-dockercfg-wk67l\""
Sep 18 18:28:30.712862 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.712735    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-46c63c\""
Sep 18 18:28:30.714440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.714418    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:28:30.828204 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.828171    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rlrd\" (UniqueName: \"kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.828466 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.828440    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.929603 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.929572    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.929702 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.929626    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rlrd\" (UniqueName: \"kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.930260 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.930242    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:30.938661 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:30.938642    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rlrd\" (UniqueName: \"kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd\") pod \"payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:31.022175 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.022154    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:31.027625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.027606    2550 scope.go:117] "RemoveContainer" containerID="b2e737773d6fba3bf245de359fece88920ff891c1996943d3b8e2d2d5e045acd"
Sep 18 18:28:31.048241 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.048218    2550 scope.go:117] "RemoveContainer" containerID="d5c60b95d86392647f732f7bbfb45fe1123c3f0a2385ab87f53d5f48c90e8b59"
Sep 18 18:28:31.316370 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.316343    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:28:31.318804 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:28:31.318772    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod394dc514_cf74_41ff_9576_61d14d012c8b.slice/crio-de606045ec3da0b23251a8282c9db569f94eb9bc6f053f1620d769f76a243774 WatchSource:0}: Error finding container de606045ec3da0b23251a8282c9db569f94eb9bc6f053f1620d769f76a243774: Status 404 returned error can't find the container with id de606045ec3da0b23251a8282c9db569f94eb9bc6f053f1620d769f76a243774
Sep 18 18:28:31.443921 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.443893    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" event={"ID":"394dc514-cf74-41ff-9576-61d14d012c8b","Type":"ContainerStarted","Data":"1717390e405230514bef93ca228030a50da892bb5c1f980ef1e0689276920f38"}
Sep 18 18:28:31.444056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.443985    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" event={"ID":"394dc514-cf74-41ff-9576-61d14d012c8b","Type":"ContainerStarted","Data":"de606045ec3da0b23251a8282c9db569f94eb9bc6f053f1620d769f76a243774"}
Sep 18 18:28:31.444056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.444014    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:28:31.468162 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:31.468119    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" podStartSLOduration=1.4681046580000001 podStartE2EDuration="1.468104658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:28:31.466403555 +0000 UTC m=+1683.153033011" watchObservedRunningTime="2026-09-18 18:28:31.468104658 +0000 UTC m=+1683.154734110"
Sep 18 18:28:34.783741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:34.783709    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:28:42.451098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:28:42.451036    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:29:04.279520 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.279485    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:29:04.282913 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.282896    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.286275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.286248    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-15e8cc\"/\"ipp-route-15e8cc-kserve-self-signed-certs\""
Sep 18 18:29:04.286401 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.286286    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-15e8cc\"/\"default-dockercfg-4zt28\""
Sep 18 18:29:04.286401 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.286299    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-15e8cc\"/\"openshift-service-ca.crt\""
Sep 18 18:29:04.286534 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.286452    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-15e8cc\"/\"kube-root-ca.crt\""
Sep 18 18:29:04.291399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.291376    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:29:04.385991 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.385967    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4n5h\" (UniqueName: \"kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.386118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.386014    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.386118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.386071    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.386118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.386113    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.386259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.386148    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.386259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.386174    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.486805 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.486782    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4n5h\" (UniqueName: \"kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.486902 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.486817    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.486902 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.486845    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487040 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487017    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487103 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487088    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487125    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487259 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487238    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487351    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.487477 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.487458    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.489158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.489139    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.489457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.489434    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.495286 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.495263    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4n5h\" (UniqueName: \"kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h\") pod \"ipp-route-15e8cc-kserve-86ddbff6c5-p8s87\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") " pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.593641 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.593586    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:04.865057 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:04.864991    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:29:04.867458 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:29:04.867432    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b584a2a_0559_4d62_a3d7_87f48b68bf25.slice/crio-8328550a602fb6b60d5dde09de7e9cac57dc8f73f0640704cd3c748293622191 WatchSource:0}: Error finding container 8328550a602fb6b60d5dde09de7e9cac57dc8f73f0640704cd3c748293622191: Status 404 returned error can't find the container with id 8328550a602fb6b60d5dde09de7e9cac57dc8f73f0640704cd3c748293622191
Sep 18 18:29:05.630659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:05.630624    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" event={"ID":"5b584a2a-0559-4d62-a3d7-87f48b68bf25","Type":"ContainerStarted","Data":"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"}
Sep 18 18:29:05.630659 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:05.630666    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" event={"ID":"5b584a2a-0559-4d62-a3d7-87f48b68bf25","Type":"ContainerStarted","Data":"8328550a602fb6b60d5dde09de7e9cac57dc8f73f0640704cd3c748293622191"}
Sep 18 18:29:05.648035 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:05.647993    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" podStartSLOduration=1.6479777759999998 podStartE2EDuration="1.647977776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:29:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:29:05.647109412 +0000 UTC m=+1717.333738878" watchObservedRunningTime="2026-09-18 18:29:05.647977776 +0000 UTC m=+1717.334607227"
Sep 18 18:29:08.081196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:08.081158    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:29:14.593922 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:14.593884    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:14.594397 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:14.593957    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:14.606553 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:14.606525    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:14.693153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:14.693121    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:29:21.889975 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:21.889925    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:29:28.411782 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:28.411753    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5_9a696c9f-849d-436a-bb32-b52faf3087cf/payload-processing/0.log"
Sep 18 18:29:29.319956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.319914    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:29:29.323301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.323278    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.325971 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.325948    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-12b560-openshift-default-dockercfg-jjlvc\""
Sep 18 18:29:29.334372 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.334349    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:29:29.399920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.399897    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.399928    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.399987    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400003    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400038 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400028    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400046    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400122    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400177 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400148    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.400274 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.400176    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67h8d\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501348 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501316    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501358    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501409    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501440    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501489    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67h8d\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501519    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501561    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501621    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501648    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.501776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501730    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.502231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501916    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.502231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.501980    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.502231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.502051    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.502399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.502381    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.503878 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.503836    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.504046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.504029    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.510253 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.510227    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.510436 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.510420    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67h8d\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d\") pod \"e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") " pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.634741 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.634682    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:29.910993 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.910967    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:29:29.913029 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:29:29.913001    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32afe2be_3ea5_4112_93fe_d77ab4bde6f5.slice/crio-adc57cad88942bc09883cd583723b9c4a518a259ceecb73a2b5b306982a25fd5 WatchSource:0}: Error finding container adc57cad88942bc09883cd583723b9c4a518a259ceecb73a2b5b306982a25fd5: Status 404 returned error can't find the container with id adc57cad88942bc09883cd583723b9c4a518a259ceecb73a2b5b306982a25fd5
Sep 18 18:29:29.915601 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.915567    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:29.915708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.915633    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:29.915708 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:29.915659    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:30.761482 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:30.761447    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerStarted","Data":"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"}
Sep 18 18:29:30.761863 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:30.761505    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerStarted","Data":"adc57cad88942bc09883cd583723b9c4a518a259ceecb73a2b5b306982a25fd5"}
Sep 18 18:29:30.783374 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:30.783329    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podStartSLOduration=1.783313758 podStartE2EDuration="1.783313758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:29:30.781466757 +0000 UTC m=+1742.468096243" watchObservedRunningTime="2026-09-18 18:29:30.783313758 +0000 UTC m=+1742.469943211"
Sep 18 18:29:31.081956 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:31.081875    2550 scope.go:117] "RemoveContainer" containerID="bcbd06c2c10054e5cdda2be1e44a81999af2056622fde0a70be0db272d13f0a7"
Sep 18 18:29:31.634877 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:31.634849    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:31.639966 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:31.639926    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:31.766897 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:31.766868    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:31.767591 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:31.767573    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:36.351547 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.351518    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:29:36.356236 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.356218    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.358512 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.358490    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-12b560\""
Sep 18 18:29:36.358780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.358768    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-12b560-dockercfg-d5wz7\""
Sep 18 18:29:36.364229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.364207    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:29:36.458892 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.458864    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqwd\" (UniqueName: \"kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.459076 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.458917    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.560385 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.560356    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqwd\" (UniqueName: \"kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.560504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.560401    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.561080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.561063    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.567932 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.567913    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqwd\" (UniqueName: \"kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd\") pod \"payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.667341 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.667281    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:36.942637 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:36.942555    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:29:36.950094 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:29:36.950067    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ccef3d_acef_4630_a78a_d0616b7e8862.slice/crio-5429acca70c769d5d6bc04ac714862a32005f1181edf034069976ffaca97475a WatchSource:0}: Error finding container 5429acca70c769d5d6bc04ac714862a32005f1181edf034069976ffaca97475a: Status 404 returned error can't find the container with id 5429acca70c769d5d6bc04ac714862a32005f1181edf034069976ffaca97475a
Sep 18 18:29:37.797397 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:37.797358    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" event={"ID":"a1ccef3d-acef-4630-a78a-d0616b7e8862","Type":"ContainerStarted","Data":"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"}
Sep 18 18:29:37.797799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:37.797420    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" event={"ID":"a1ccef3d-acef-4630-a78a-d0616b7e8862","Type":"ContainerStarted","Data":"5429acca70c769d5d6bc04ac714862a32005f1181edf034069976ffaca97475a"}
Sep 18 18:29:37.797799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:37.797456    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:37.813114 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:37.813068    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" podStartSLOduration=1.813053303 podStartE2EDuration="1.813053303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:29:37.810538632 +0000 UTC m=+1749.497168083" watchObservedRunningTime="2026-09-18 18:29:37.813053303 +0000 UTC m=+1749.499682755"
Sep 18 18:29:43.286176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:43.286141    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:29:48.803576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:48.803543    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:29:53.905679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.905645    2550 generic.go:356] "Generic (PLEG): container finished" podID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerID="8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103" exitCode=0
Sep 18 18:29:53.906137 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.905724    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerDied","Data":"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"}
Sep 18 18:29:53.906196 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.906188    2550 scope.go:117] "RemoveContainer" containerID="8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"
Sep 18 18:29:53.907327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.907296    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:53.907457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.907381    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:53.907457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:53.907428    2550 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 18:29:54.917081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:54.917050    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerStarted","Data":"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"}
Sep 18 18:29:55.635854 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:55.635825    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:29:55.636120 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:55.636091    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 18 18:29:55.636180 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:55.636135    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 18 18:29:56.635309 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:56.635274    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 18 18:29:56.635697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:56.635332    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 18 18:29:57.635345 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:57.635313    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 18 18:29:57.635718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:57.635365    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 18 18:29:58.335824 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.335790    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"]
Sep 18 18:29:58.338872 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.338855    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:29:58.341620 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.341430    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:29:58.341785 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.341762    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:29:58.347561 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.347534    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"]
Sep 18 18:29:58.472642 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.472614    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m\" (UID: \"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:29:58.573092 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.573058    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m\" (UID: \"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:29:58.573663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.573644    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m\" (UID: \"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:29:58.635344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.635281    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 18 18:29:58.635344 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.635327    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 18 18:29:58.653984 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.653957    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:29:58.930346 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:58.930322    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"]
Sep 18 18:29:58.932295 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:29:58.932269    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6163dbf_ecdc_4d8e_b48f_87f97296bdcb.slice/crio-78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0 WatchSource:0}: Error finding container 78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0: Status 404 returned error can't find the container with id 78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0
Sep 18 18:29:59.636004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:59.635908    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Sep 18 18:29:59.636004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:59.635984    2550 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Sep 18 18:29:59.954080 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:59.954040    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m" event={"ID":"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb","Type":"ContainerStarted","Data":"8b51cc18acc783eab10578e1c01ec845045d8b16ec1b60695d236fa18b402854"}
Sep 18 18:29:59.954258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:59.954131    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m" event={"ID":"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb","Type":"ContainerStarted","Data":"78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0"}
Sep 18 18:29:59.967772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:29:59.967664    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m" podStartSLOduration=1.9676457219999999 podStartE2EDuration="1.967645722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:29:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:29:59.966839372 +0000 UTC m=+1771.653468825" watchObservedRunningTime="2026-09-18 18:29:59.967645722 +0000 UTC m=+1771.654275175"
Sep 18 18:30:00.138998 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.138949    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"]
Sep 18 18:30:00.142183 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.142162    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"]
Sep 18 18:30:00.142320 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.142294    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"
Sep 18 18:30:00.145029 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.145008    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"]
Sep 18 18:30:00.145149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.145126    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"
Sep 18 18:30:00.147806 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.147785    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"
Sep 18 18:30:00.160793 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.160768    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"]
Sep 18 18:30:00.163576 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.161408    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"
Sep 18 18:30:00.163762 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.163738    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"]
Sep 18 18:30:00.165887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.165858    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"]
Sep 18 18:30:00.179612 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.179374    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"
Sep 18 18:30:00.216014 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.215972    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"
Sep 18 18:30:00.456282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.456219    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"]
Sep 18 18:30:00.459398 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:30:00.459370    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fee629_ef1d_409d_9ddb_6c66f2ce256a.slice/crio-7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194 WatchSource:0}: Error finding container 7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194: Status 404 returned error can't find the container with id 7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194
Sep 18 18:30:00.467039 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.467020    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"]
Sep 18 18:30:00.471309 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:30:00.471286    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac99cc7e_f468_45b9_bcc2_c20161ab6a18.slice/crio-dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88 WatchSource:0}: Error finding container dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88: Status 404 returned error can't find the container with id dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88
Sep 18 18:30:00.506271 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.506249    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"]
Sep 18 18:30:00.507853 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:30:00.507834    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d6a7ba_f895_46ce_a8a4_828b078f5fe8.slice/crio-c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe WatchSource:0}: Error finding container c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe: Status 404 returned error can't find the container with id c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe
Sep 18 18:30:00.639447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.639422    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:30:00.639782 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.639677    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:30:00.640570 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.640556    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:30:00.963001 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.962967    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf" event={"ID":"ac99cc7e-f468-45b9-bcc2-c20161ab6a18","Type":"ContainerStarted","Data":"70dedd3d89bf53884610db581007021cb3cf1413de3fd7f3225dd35a7b2fa336"}
Sep 18 18:30:00.963153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.963020    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf" event={"ID":"ac99cc7e-f468-45b9-bcc2-c20161ab6a18","Type":"ContainerStarted","Data":"dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88"}
Sep 18 18:30:00.964584 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.964556    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw" event={"ID":"87fee629-ef1d-409d-9ddb-6c66f2ce256a","Type":"ContainerStarted","Data":"54cf5a6bfbfdf5754878b78c1df658ae5347dc70fc6f05fb74825e36137cbb9f"}
Sep 18 18:30:00.964688 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.964603    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw" event={"ID":"87fee629-ef1d-409d-9ddb-6c66f2ce256a","Type":"ContainerStarted","Data":"7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194"}
Sep 18 18:30:00.966042 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.966020    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr" event={"ID":"86d6a7ba-f895-46ce-a8a4-828b078f5fe8","Type":"ContainerStarted","Data":"f60abd9950e2243ef239cc2d5595467e25eb8e992836eab10aa6eaa728f9563f"}
Sep 18 18:30:00.966126 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.966061    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr" event={"ID":"86d6a7ba-f895-46ce-a8a4-828b078f5fe8","Type":"ContainerStarted","Data":"c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe"}
Sep 18 18:30:00.967533 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.967515    2550 generic.go:356] "Generic (PLEG): container finished" podID="c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" containerID="8b51cc18acc783eab10578e1c01ec845045d8b16ec1b60695d236fa18b402854" exitCode=0
Sep 18 18:30:00.967618 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.967577    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m" event={"ID":"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb","Type":"ContainerDied","Data":"8b51cc18acc783eab10578e1c01ec845045d8b16ec1b60695d236fa18b402854"}
Sep 18 18:30:00.979400 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.979364    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf" podStartSLOduration=0.979351951 podStartE2EDuration="979.351951ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:30:00.978468553 +0000 UTC m=+1772.665098005" watchObservedRunningTime="2026-09-18 18:30:00.979351951 +0000 UTC m=+1772.665981402"
Sep 18 18:30:00.999461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:00.999398    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw" podStartSLOduration=0.999385949 podStartE2EDuration="999.385949ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:30:00.997007557 +0000 UTC m=+1772.683637008" watchObservedRunningTime="2026-09-18 18:30:00.999385949 +0000 UTC m=+1772.686015401"
Sep 18 18:30:01.024912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.024874    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr" podStartSLOduration=1.024862256 podStartE2EDuration="1.024862256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:30:01.023867149 +0000 UTC m=+1772.710496631" watchObservedRunningTime="2026-09-18 18:30:01.024862256 +0000 UTC m=+1772.711491708"
Sep 18 18:30:01.974662 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.974627    2550 generic.go:356] "Generic (PLEG): container finished" podID="ac99cc7e-f468-45b9-bcc2-c20161ab6a18" containerID="70dedd3d89bf53884610db581007021cb3cf1413de3fd7f3225dd35a7b2fa336" exitCode=0
Sep 18 18:30:01.975033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.974682    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf" event={"ID":"ac99cc7e-f468-45b9-bcc2-c20161ab6a18","Type":"ContainerDied","Data":"70dedd3d89bf53884610db581007021cb3cf1413de3fd7f3225dd35a7b2fa336"}
Sep 18 18:30:01.976338 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.976315    2550 generic.go:356] "Generic (PLEG): container finished" podID="87fee629-ef1d-409d-9ddb-6c66f2ce256a" containerID="54cf5a6bfbfdf5754878b78c1df658ae5347dc70fc6f05fb74825e36137cbb9f" exitCode=0
Sep 18 18:30:01.976451 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.976366    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw" event={"ID":"87fee629-ef1d-409d-9ddb-6c66f2ce256a","Type":"ContainerDied","Data":"54cf5a6bfbfdf5754878b78c1df658ae5347dc70fc6f05fb74825e36137cbb9f"}
Sep 18 18:30:01.978026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.978006    2550 generic.go:356] "Generic (PLEG): container finished" podID="86d6a7ba-f895-46ce-a8a4-828b078f5fe8" containerID="f60abd9950e2243ef239cc2d5595467e25eb8e992836eab10aa6eaa728f9563f" exitCode=0
Sep 18 18:30:01.978158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:01.978134    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr" event={"ID":"86d6a7ba-f895-46ce-a8a4-828b078f5fe8","Type":"ContainerDied","Data":"f60abd9950e2243ef239cc2d5595467e25eb8e992836eab10aa6eaa728f9563f"}
Sep 18 18:30:02.130906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.130878    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:30:02.207769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.207745    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca\") pod \"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb\" (UID: \"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb\") "
Sep 18 18:30:02.208069 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.208047    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" (UID: "c6163dbf-ecdc-4d8e-b48f-87f97296bdcb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:02.308969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.308927    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:02.985335 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.985268    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"
Sep 18 18:30:02.985335 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.985308    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m" event={"ID":"c6163dbf-ecdc-4d8e-b48f-87f97296bdcb","Type":"ContainerDied","Data":"78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0"}
Sep 18 18:30:02.985808 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:02.985366    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78493d177a799e37bbd11df105922eeeb0156c8fbbfedecada6cb89daff987f0"
Sep 18 18:30:03.129046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.129012    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"
Sep 18 18:30:03.170357 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.170333    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"
Sep 18 18:30:03.173876 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.173858    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"
Sep 18 18:30:03.995044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.995014    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"
Sep 18 18:30:03.995504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.995042    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf" event={"ID":"ac99cc7e-f468-45b9-bcc2-c20161ab6a18","Type":"ContainerDied","Data":"dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88"}
Sep 18 18:30:03.995504 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.995084    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd4003cafdca311e8dd0be20930b10d03d90277bec242b296b43311b5a91fd88"
Sep 18 18:30:03.996382 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.996360    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"
Sep 18 18:30:03.996473 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.996385    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw" event={"ID":"87fee629-ef1d-409d-9ddb-6c66f2ce256a","Type":"ContainerDied","Data":"7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194"}
Sep 18 18:30:03.996473 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.996425    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d19886fc480a5850f9d242f0f9c13de5528450933c64d60d8f81227f2e94194"
Sep 18 18:30:03.997658 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.997642    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"
Sep 18 18:30:03.997763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.997644    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr" event={"ID":"86d6a7ba-f895-46ce-a8a4-828b078f5fe8","Type":"ContainerDied","Data":"c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe"}
Sep 18 18:30:03.997763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:03.997746    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c50ac058f480ad6380bd12cb959d0b362c33af4203ebcc4fee2cb658090eaebe"
Sep 18 18:30:04.737093 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:04.737063    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:30:04.737301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:04.737273    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" podUID="a1ccef3d-acef-4630-a78a-d0616b7e8862" containerName="payload-pre-processing" containerID="cri-o://517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a" gracePeriod=30
Sep 18 18:30:04.952596 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:04.952560    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"]
Sep 18 18:30:04.956000 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:04.955972    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-12b560-r5g8m"]
Sep 18 18:30:04.983720 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:04.983699    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:30:05.006239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.006183    2550 generic.go:356] "Generic (PLEG): container finished" podID="a1ccef3d-acef-4630-a78a-d0616b7e8862" containerID="517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a" exitCode=0
Sep 18 18:30:05.006239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.006214    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" event={"ID":"a1ccef3d-acef-4630-a78a-d0616b7e8862","Type":"ContainerDied","Data":"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"}
Sep 18 18:30:05.006239 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.006224    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"
Sep 18 18:30:05.006664 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.006251    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz" event={"ID":"a1ccef3d-acef-4630-a78a-d0616b7e8862","Type":"ContainerDied","Data":"5429acca70c769d5d6bc04ac714862a32005f1181edf034069976ffaca97475a"}
Sep 18 18:30:05.006664 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.006274    2550 scope.go:117] "RemoveContainer" containerID="517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"
Sep 18 18:30:05.014923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.014901    2550 scope.go:117] "RemoveContainer" containerID="517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"
Sep 18 18:30:05.015225 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:30:05.015207    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a\": container with ID starting with 517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a not found: ID does not exist" containerID="517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"
Sep 18 18:30:05.015294 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.015237    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a"} err="failed to get container status \"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a\": rpc error: code = NotFound desc = could not find container \"517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a\": container with ID starting with 517b09106eb764a23e1e7b32e0b9d448f5910514f807fe465b7a14a77112753a not found: ID does not exist"
Sep 18 18:30:05.028680 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.028655    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume\") pod \"a1ccef3d-acef-4630-a78a-d0616b7e8862\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") "
Sep 18 18:30:05.028766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.028742    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbqwd\" (UniqueName: \"kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd\") pod \"a1ccef3d-acef-4630-a78a-d0616b7e8862\" (UID: \"a1ccef3d-acef-4630-a78a-d0616b7e8862\") "
Sep 18 18:30:05.028982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.028963    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1ccef3d-acef-4630-a78a-d0616b7e8862" (UID: "a1ccef3d-acef-4630-a78a-d0616b7e8862"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:05.029072 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.029054    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1ccef3d-acef-4630-a78a-d0616b7e8862-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.030616 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.030598    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd" (OuterVolumeSpecName: "kube-api-access-jbqwd") pod "a1ccef3d-acef-4630-a78a-d0616b7e8862" (UID: "a1ccef3d-acef-4630-a78a-d0616b7e8862"). InnerVolumeSpecName "kube-api-access-jbqwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:05.129699 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.129667    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbqwd\" (UniqueName: \"kubernetes.io/projected/a1ccef3d-acef-4630-a78a-d0616b7e8862-kube-api-access-jbqwd\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:05.326553 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.326528    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:30:05.330401 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:05.330380    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-12b560-7fb98d64d6-6kgxz"]
Sep 18 18:30:06.706789 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:06.706757    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ccef3d-acef-4630-a78a-d0616b7e8862" path="/var/lib/kubelet/pods/a1ccef3d-acef-4630-a78a-d0616b7e8862/volumes"
Sep 18 18:30:06.707185 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:06.707090    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" path="/var/lib/kubelet/pods/c6163dbf-ecdc-4d8e-b48f-87f97296bdcb/volumes"
Sep 18 18:30:09.981086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:09.981042    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:30:11.394046 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:11.394015    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:30:11.394450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:11.394208    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" containerID="cri-o://ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3" gracePeriod=30
Sep 18 18:30:15.642261 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:15.642181    2550 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 18:30:15.642261 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:15.642236    2550 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 18:30:16.638842 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.638819    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:30:16.724835 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724770    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.724835 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724797    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.724835 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724828    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724849    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724883    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724922    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67h8d\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724965    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.724995    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725062    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725178    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo\") pod \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\" (UID: \"32afe2be-3ea5-4112-93fe-d77ab4bde6f5\") "
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725256    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:30:16.725389 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725351    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725431    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725626    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725645    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725709    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725726    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.726067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.725756    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:30:16.727146 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.727116    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:30:16.727268 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.727165    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:16.727540 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.727518    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d" (OuterVolumeSpecName: "kube-api-access-67h8d") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "kube-api-access-67h8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:16.727587 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.727543    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "32afe2be-3ea5-4112-93fe-d77ab4bde6f5" (UID: "32afe2be-3ea5-4112-93fe-d77ab4bde6f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:30:16.826293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.826268    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.826293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.826290    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.826448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.826299    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.826448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.826308    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67h8d\" (UniqueName: \"kubernetes.io/projected/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-kube-api-access-67h8d\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:16.826448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:16.826316    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32afe2be-3ea5-4112-93fe-d77ab4bde6f5-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:17.074379 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.074339    2550 generic.go:356] "Generic (PLEG): container finished" podID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerID="ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3" exitCode=0
Sep 18 18:30:17.074565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.074386    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"
Sep 18 18:30:17.074565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.074419    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerDied","Data":"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"}
Sep 18 18:30:17.074565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.074469    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk" event={"ID":"32afe2be-3ea5-4112-93fe-d77ab4bde6f5","Type":"ContainerDied","Data":"adc57cad88942bc09883cd583723b9c4a518a259ceecb73a2b5b306982a25fd5"}
Sep 18 18:30:17.074565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.074484    2550 scope.go:117] "RemoveContainer" containerID="ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"
Sep 18 18:30:17.084084 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.084069    2550 scope.go:117] "RemoveContainer" containerID="8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"
Sep 18 18:30:17.092029 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.092013    2550 scope.go:117] "RemoveContainer" containerID="ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"
Sep 18 18:30:17.092293 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:30:17.092271    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3\": container with ID starting with ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3 not found: ID does not exist" containerID="ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"
Sep 18 18:30:17.092379 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.092302    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3"} err="failed to get container status \"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3\": rpc error: code = NotFound desc = could not find container \"ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3\": container with ID starting with ae8ba67355bc324a43f68c546f1610a1d585a4ca8f42b9f29faab9a0923c35d3 not found: ID does not exist"
Sep 18 18:30:17.092379 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.092320    2550 scope.go:117] "RemoveContainer" containerID="8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"
Sep 18 18:30:17.092565 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:30:17.092540    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103\": container with ID starting with 8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103 not found: ID does not exist" containerID="8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"
Sep 18 18:30:17.092631 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.092574    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103"} err="failed to get container status \"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103\": rpc error: code = NotFound desc = could not find container \"8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103\": container with ID starting with 8fd1ff23092c382ab05c615f5910f0047c957161c69e5d681bcc22bc64527103 not found: ID does not exist"
Sep 18 18:30:17.096359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.096338    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:30:17.100016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:17.099995    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-12b560-openshift-default-65db5d8c5b-7sxmk"]
Sep 18 18:30:18.706376 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:18.706344    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" path="/var/lib/kubelet/pods/32afe2be-3ea5-4112-93fe-d77ab4bde6f5/volumes"
Sep 18 18:30:25.450176 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450134    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"]
Sep 18 18:30:25.450652 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450632    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac99cc7e-f468-45b9-bcc2-c20161ab6a18" containerName="cleanup"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450656    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac99cc7e-f468-45b9-bcc2-c20161ab6a18" containerName="cleanup"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450698    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" containerName="revoke-keys"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450707    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" containerName="revoke-keys"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450726    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450735    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:30:25.450749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450744    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87fee629-ef1d-409d-9ddb-6c66f2ce256a" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450754    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="87fee629-ef1d-409d-9ddb-6c66f2ce256a" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450769    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450779    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450790    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1ccef3d-acef-4630-a78a-d0616b7e8862" containerName="payload-pre-processing"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450799    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ccef3d-acef-4630-a78a-d0616b7e8862" containerName="payload-pre-processing"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450809    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d6a7ba-f895-46ce-a8a4-828b078f5fe8" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450817    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d6a7ba-f895-46ce-a8a4-828b078f5fe8" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450932    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d6a7ba-f895-46ce-a8a4-828b078f5fe8" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450963    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac99cc7e-f468-45b9-bcc2-c20161ab6a18" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450973    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1ccef3d-acef-4630-a78a-d0616b7e8862" containerName="payload-pre-processing"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450984    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="87fee629-ef1d-409d-9ddb-6c66f2ce256a" containerName="cleanup"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.450993    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:30:25.451086 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.451001    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6163dbf-ecdc-4d8e-b48f-87f97296bdcb" containerName="revoke-keys"
Sep 18 18:30:25.454102 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.454079    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:25.456485 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.456454    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:30:25.457432 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.457404    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:30:25.460243 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.460216    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"]
Sep 18 18:30:25.499365 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.499337    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f\" (UID: \"f23e92db-d81c-4587-91e3-db1ae77bc7cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:25.600168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.600133    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f\" (UID: \"f23e92db-d81c-4587-91e3-db1ae77bc7cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:25.600678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.600661    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f\" (UID: \"f23e92db-d81c-4587-91e3-db1ae77bc7cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:25.768711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:25.768690    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:26.041100 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:26.041024    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"]
Sep 18 18:30:26.044133 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:30:26.044108    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23e92db_d81c_4587_91e3_db1ae77bc7cb.slice/crio-38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89 WatchSource:0}: Error finding container 38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89: Status 404 returned error can't find the container with id 38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89
Sep 18 18:30:26.122488 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:26.122460    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f" event={"ID":"f23e92db-d81c-4587-91e3-db1ae77bc7cb","Type":"ContainerStarted","Data":"38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89"}
Sep 18 18:30:27.130340 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:27.130305    2550 generic.go:356] "Generic (PLEG): container finished" podID="f23e92db-d81c-4587-91e3-db1ae77bc7cb" containerID="3e391bb2c68743dca83851e422bd2ca6bf3ed1aa7ec94e20a9482e90bf9cf959" exitCode=0
Sep 18 18:30:27.130685 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:27.130386    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f" event={"ID":"f23e92db-d81c-4587-91e3-db1ae77bc7cb","Type":"ContainerDied","Data":"3e391bb2c68743dca83851e422bd2ca6bf3ed1aa7ec94e20a9482e90bf9cf959"}
Sep 18 18:30:28.263141 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:28.263118    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:28.322099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:28.322072    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca\") pod \"f23e92db-d81c-4587-91e3-db1ae77bc7cb\" (UID: \"f23e92db-d81c-4587-91e3-db1ae77bc7cb\") "
Sep 18 18:30:28.322411 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:28.322387    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f23e92db-d81c-4587-91e3-db1ae77bc7cb" (UID: "f23e92db-d81c-4587-91e3-db1ae77bc7cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:28.422838 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:28.422781    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f23e92db-d81c-4587-91e3-db1ae77bc7cb-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:29.142416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:29.142377    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f" event={"ID":"f23e92db-d81c-4587-91e3-db1ae77bc7cb","Type":"ContainerDied","Data":"38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89"}
Sep 18 18:30:29.142416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:29.142413    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"
Sep 18 18:30:29.142625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:29.142427    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38e8b3e242f1b8bcb1f16b830b06301510b4bdf570ccbd2b631ae410afb5cf89"
Sep 18 18:30:30.812321 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:30.812293    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:30:30.816983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:30.816959    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:30:31.100709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:31.100635    2550 scope.go:117] "RemoveContainer" containerID="d650985a8a1a93c820f74d714fc9e7b441360c68ec2428cbd83f151ff513055b"
Sep 18 18:30:38.577065 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:38.577031    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:30:40.957150 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.957117    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"]
Sep 18 18:30:40.959724 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.959699    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-15e8cc-29829270-8r4mw"]
Sep 18 18:30:40.983221 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.983191    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:30:40.983464 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.983409    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" podUID="9a696c9f-849d-436a-bb32-b52faf3087cf" containerName="payload-processing" containerID="cri-o://c788679df34f945313aa4e3f2c2156bff00222e4a4b8360c23a48f6b901e645a" gracePeriod=30
Sep 18 18:30:40.990951 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.990914    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:30:40.991179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:40.991150    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" podUID="df72e136-c898-4cd2-a2d8-d0e34e360085" containerName="payload-pre-processing" containerID="cri-o://a470758cf56769c904b68629ceb2827bb837796364f69311e1ca62c33d19a285" gracePeriod=30
Sep 18 18:30:41.208929 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.208845    2550 generic.go:356] "Generic (PLEG): container finished" podID="df72e136-c898-4cd2-a2d8-d0e34e360085" containerID="a470758cf56769c904b68629ceb2827bb837796364f69311e1ca62c33d19a285" exitCode=0
Sep 18 18:30:41.209098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.208926    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" event={"ID":"df72e136-c898-4cd2-a2d8-d0e34e360085","Type":"ContainerDied","Data":"a470758cf56769c904b68629ceb2827bb837796364f69311e1ca62c33d19a285"}
Sep 18 18:30:41.210890 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.210865    2550 generic.go:356] "Generic (PLEG): container finished" podID="9a696c9f-849d-436a-bb32-b52faf3087cf" containerID="c788679df34f945313aa4e3f2c2156bff00222e4a4b8360c23a48f6b901e645a" exitCode=0
Sep 18 18:30:41.211036 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.210962    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" event={"ID":"9a696c9f-849d-436a-bb32-b52faf3087cf","Type":"ContainerDied","Data":"c788679df34f945313aa4e3f2c2156bff00222e4a4b8360c23a48f6b901e645a"}
Sep 18 18:30:41.215409 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.215387    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"]
Sep 18 18:30:41.231571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.231530    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-15e8cc-zxg6f"]
Sep 18 18:30:41.258767 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.258747    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:30:41.261593 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.261576    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:30:41.331171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331151    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clfz6\" (UniqueName: \"kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6\") pod \"9a696c9f-849d-436a-bb32-b52faf3087cf\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") "
Sep 18 18:30:41.331288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331188    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bjbc\" (UniqueName: \"kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc\") pod \"df72e136-c898-4cd2-a2d8-d0e34e360085\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") "
Sep 18 18:30:41.331288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331230    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume\") pod \"df72e136-c898-4cd2-a2d8-d0e34e360085\" (UID: \"df72e136-c898-4cd2-a2d8-d0e34e360085\") "
Sep 18 18:30:41.331288 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331276    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume\") pod \"9a696c9f-849d-436a-bb32-b52faf3087cf\" (UID: \"9a696c9f-849d-436a-bb32-b52faf3087cf\") "
Sep 18 18:30:41.331676 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331650    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df72e136-c898-4cd2-a2d8-d0e34e360085" (UID: "df72e136-c898-4cd2-a2d8-d0e34e360085"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:41.331772 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.331679    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a696c9f-849d-436a-bb32-b52faf3087cf" (UID: "9a696c9f-849d-436a-bb32-b52faf3087cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:30:41.333454 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.333426    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc" (OuterVolumeSpecName: "kube-api-access-7bjbc") pod "df72e136-c898-4cd2-a2d8-d0e34e360085" (UID: "df72e136-c898-4cd2-a2d8-d0e34e360085"). InnerVolumeSpecName "kube-api-access-7bjbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:41.333715 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.333690    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6" (OuterVolumeSpecName: "kube-api-access-clfz6") pod "9a696c9f-849d-436a-bb32-b52faf3087cf" (UID: "9a696c9f-849d-436a-bb32-b52faf3087cf"). InnerVolumeSpecName "kube-api-access-clfz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:30:41.432373 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.432347    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bjbc\" (UniqueName: \"kubernetes.io/projected/df72e136-c898-4cd2-a2d8-d0e34e360085-kube-api-access-7bjbc\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:41.432373 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.432371    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df72e136-c898-4cd2-a2d8-d0e34e360085-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:41.432508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.432384    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a696c9f-849d-436a-bb32-b52faf3087cf-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:41.432508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:41.432396    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfz6\" (UniqueName: \"kubernetes.io/projected/9a696c9f-849d-436a-bb32-b52faf3087cf-kube-api-access-clfz6\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:30:42.217726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.217698    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"
Sep 18 18:30:42.217726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.217714    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c" event={"ID":"df72e136-c898-4cd2-a2d8-d0e34e360085","Type":"ContainerDied","Data":"2c798fd357f35850971b6c948e8d22b9de61c4bd9595ed5c9ae024ce3c889890"}
Sep 18 18:30:42.218254 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.217773    2550 scope.go:117] "RemoveContainer" containerID="a470758cf56769c904b68629ceb2827bb837796364f69311e1ca62c33d19a285"
Sep 18 18:30:42.219143 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.219112    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"
Sep 18 18:30:42.219143 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.219133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5" event={"ID":"9a696c9f-849d-436a-bb32-b52faf3087cf","Type":"ContainerDied","Data":"cf3ac64684ae2c92e91dd3e113b48127c03237372c7f476cc32425623a81e02a"}
Sep 18 18:30:42.227432 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.227148    2550 scope.go:117] "RemoveContainer" containerID="c788679df34f945313aa4e3f2c2156bff00222e4a4b8360c23a48f6b901e645a"
Sep 18 18:30:42.239769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.239746    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:30:42.246657 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.246637    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-15e8cc-746f8cfd78-f777c"]
Sep 18 18:30:42.256959 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.256914    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:30:42.258747 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.258724    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-15e8cc-569fd84d6c-drdm5"]
Sep 18 18:30:42.706013 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.705977    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87fee629-ef1d-409d-9ddb-6c66f2ce256a" path="/var/lib/kubelet/pods/87fee629-ef1d-409d-9ddb-6c66f2ce256a/volumes"
Sep 18 18:30:42.706318 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.706303    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a696c9f-849d-436a-bb32-b52faf3087cf" path="/var/lib/kubelet/pods/9a696c9f-849d-436a-bb32-b52faf3087cf/volumes"
Sep 18 18:30:42.706640 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.706628    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df72e136-c898-4cd2-a2d8-d0e34e360085" path="/var/lib/kubelet/pods/df72e136-c898-4cd2-a2d8-d0e34e360085/volumes"
Sep 18 18:30:42.706922 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:42.706910    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f23e92db-d81c-4587-91e3-db1ae77bc7cb" path="/var/lib/kubelet/pods/f23e92db-d81c-4587-91e3-db1ae77bc7cb/volumes"
Sep 18 18:30:46.620280 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:46.620248    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:30:51.579440 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:30:51.579402    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:31:01.639399 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.639333    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" podUID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" containerName="main" containerID="cri-o://1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185" gracePeriod=45
Sep 18 18:31:01.890949 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.890873    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:31:01.907098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907073    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907208 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907117    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907208 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907189    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907218    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4n5h\" (UniqueName: \"kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907280    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907339 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907306    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home\") pod \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\" (UID: \"5b584a2a-0559-4d62-a3d7-87f48b68bf25\") "
Sep 18 18:31:01.907482 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907423    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:31:01.907597 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907557    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:31:01.907718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907637    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:01.907718 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.907654    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home" (OuterVolumeSpecName: "home") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:31:01.909839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.909806    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h" (OuterVolumeSpecName: "kube-api-access-d4n5h") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "kube-api-access-d4n5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:31:01.909962 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.909919    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:31:01.910019 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:01.910004    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm" (OuterVolumeSpecName: "dshm") pod "5b584a2a-0559-4d62-a3d7-87f48b68bf25" (UID: "5b584a2a-0559-4d62-a3d7-87f48b68bf25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:31:02.009059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.009026    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b584a2a-0559-4d62-a3d7-87f48b68bf25-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:02.009059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.009050    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:02.009059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.009059    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4n5h\" (UniqueName: \"kubernetes.io/projected/5b584a2a-0559-4d62-a3d7-87f48b68bf25-kube-api-access-d4n5h\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:02.009059 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.009068    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:02.009310 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.009076    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b584a2a-0559-4d62-a3d7-87f48b68bf25-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:02.326403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.326364    2550 generic.go:356] "Generic (PLEG): container finished" podID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" containerID="1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185" exitCode=0
Sep 18 18:31:02.326565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.326407    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"
Sep 18 18:31:02.326565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.326452    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" event={"ID":"5b584a2a-0559-4d62-a3d7-87f48b68bf25","Type":"ContainerDied","Data":"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"}
Sep 18 18:31:02.326565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.326493    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87" event={"ID":"5b584a2a-0559-4d62-a3d7-87f48b68bf25","Type":"ContainerDied","Data":"8328550a602fb6b60d5dde09de7e9cac57dc8f73f0640704cd3c748293622191"}
Sep 18 18:31:02.326565 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.326514    2550 scope.go:117] "RemoveContainer" containerID="1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"
Sep 18 18:31:02.335655 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.335638    2550 scope.go:117] "RemoveContainer" containerID="1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"
Sep 18 18:31:02.335914 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:31:02.335896    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185\": container with ID starting with 1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185 not found: ID does not exist" containerID="1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"
Sep 18 18:31:02.335996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.335926    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185"} err="failed to get container status \"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185\": rpc error: code = NotFound desc = could not find container \"1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185\": container with ID starting with 1b55da4aed4dcb87faaaf511ee8f767a7e5bd290e1827a2d522fee4789c60185 not found: ID does not exist"
Sep 18 18:31:02.346646 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.346621    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:31:02.350678 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.350656    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-15e8cc/ipp-route-15e8cc-kserve-86ddbff6c5-p8s87"]
Sep 18 18:31:02.707158 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:02.707091    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" path="/var/lib/kubelet/pods/5b584a2a-0559-4d62-a3d7-87f48b68bf25/volumes"
Sep 18 18:31:29.823168 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823130    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"]
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823485    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a696c9f-849d-436a-bb32-b52faf3087cf" containerName="payload-processing"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823495    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a696c9f-849d-436a-bb32-b52faf3087cf" containerName="payload-processing"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823507    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f23e92db-d81c-4587-91e3-db1ae77bc7cb" containerName="revoke-keys"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823513    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="f23e92db-d81c-4587-91e3-db1ae77bc7cb" containerName="revoke-keys"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823523    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df72e136-c898-4cd2-a2d8-d0e34e360085" containerName="payload-pre-processing"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823528    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="df72e136-c898-4cd2-a2d8-d0e34e360085" containerName="payload-pre-processing"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823536    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" containerName="main"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823541    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" containerName="main"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823594    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="32afe2be-3ea5-4112-93fe-d77ab4bde6f5" containerName="istio-proxy"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823605    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a696c9f-849d-436a-bb32-b52faf3087cf" containerName="payload-processing"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823611    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="f23e92db-d81c-4587-91e3-db1ae77bc7cb" containerName="revoke-keys"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823617    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b584a2a-0559-4d62-a3d7-87f48b68bf25" containerName="main"
Sep 18 18:31:29.823625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.823625    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="df72e136-c898-4cd2-a2d8-d0e34e360085" containerName="payload-pre-processing"
Sep 18 18:31:29.826562 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.826544    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:29.828781 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.828753    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:31:29.828996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.828981    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:31:29.834709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.834684    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"]
Sep 18 18:31:29.928276 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:29.928238    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2\" (UID: \"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:30.029390 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:30.029357    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2\" (UID: \"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:30.029982 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:30.029930    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2\" (UID: \"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:30.138448 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:30.138363    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:30.409433 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:30.409409    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"]
Sep 18 18:31:30.411786 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:31:30.411757    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac2a71c_fdb1_4d98_980a_03dfd5faa5e9.slice/crio-f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e WatchSource:0}: Error finding container f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e: Status 404 returned error can't find the container with id f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e
Sep 18 18:31:30.476407 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:30.476377    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2" event={"ID":"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9","Type":"ContainerStarted","Data":"f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e"}
Sep 18 18:31:31.483301 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:31.483200    2550 generic.go:356] "Generic (PLEG): container finished" podID="bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" containerID="502caf2c389085411b57bd64dbf98cd18b42bda5a997cebc4096f5a57059902a" exitCode=0
Sep 18 18:31:31.488814 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:31.483871    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2" event={"ID":"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9","Type":"ContainerDied","Data":"502caf2c389085411b57bd64dbf98cd18b42bda5a997cebc4096f5a57059902a"}
Sep 18 18:31:32.622359 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:32.622331    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:32.755441 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:32.755357    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca\") pod \"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9\" (UID: \"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9\") "
Sep 18 18:31:32.755769 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:32.755749    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" (UID: "bac2a71c-fdb1-4d98-980a-03dfd5faa5e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:32.856906 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:32.856868    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:33.495282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:33.495250    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"
Sep 18 18:31:33.495498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:33.495244    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2" event={"ID":"bac2a71c-fdb1-4d98-980a-03dfd5faa5e9","Type":"ContainerDied","Data":"f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e"}
Sep 18 18:31:33.495498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:33.495378    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f268178d314229a3956f82c2a9105eaab14045345f707c26aa600c7733fd472e"
Sep 18 18:31:40.344898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.344865    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:31:40.345431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.345100    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" podUID="394dc514-cf74-41ff-9576-61d14d012c8b" containerName="payload-pre-processing" containerID="cri-o://1717390e405230514bef93ca228030a50da892bb5c1f980ef1e0689276920f38" gracePeriod=30
Sep 18 18:31:40.524275 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.524195    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"]
Sep 18 18:31:40.526222 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.526195    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-46c63c-p5ng2"]
Sep 18 18:31:40.533018 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.532983    2550 generic.go:356] "Generic (PLEG): container finished" podID="394dc514-cf74-41ff-9576-61d14d012c8b" containerID="1717390e405230514bef93ca228030a50da892bb5c1f980ef1e0689276920f38" exitCode=0
Sep 18 18:31:40.533170 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.533058    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" event={"ID":"394dc514-cf74-41ff-9576-61d14d012c8b","Type":"ContainerDied","Data":"1717390e405230514bef93ca228030a50da892bb5c1f980ef1e0689276920f38"}
Sep 18 18:31:40.586921 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.586900    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:31:40.706807 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.706726    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" path="/var/lib/kubelet/pods/bac2a71c-fdb1-4d98-980a-03dfd5faa5e9/volumes"
Sep 18 18:31:40.724831 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.724805    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rlrd\" (UniqueName: \"kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd\") pod \"394dc514-cf74-41ff-9576-61d14d012c8b\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") "
Sep 18 18:31:40.725024 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.724919    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume\") pod \"394dc514-cf74-41ff-9576-61d14d012c8b\" (UID: \"394dc514-cf74-41ff-9576-61d14d012c8b\") "
Sep 18 18:31:40.725257 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.725232    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "394dc514-cf74-41ff-9576-61d14d012c8b" (UID: "394dc514-cf74-41ff-9576-61d14d012c8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:40.726857 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.726836    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd" (OuterVolumeSpecName: "kube-api-access-5rlrd") pod "394dc514-cf74-41ff-9576-61d14d012c8b" (UID: "394dc514-cf74-41ff-9576-61d14d012c8b"). InnerVolumeSpecName "kube-api-access-5rlrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:31:40.826118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.826087    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rlrd\" (UniqueName: \"kubernetes.io/projected/394dc514-cf74-41ff-9576-61d14d012c8b-kube-api-access-5rlrd\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:40.826118 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:40.826116    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/394dc514-cf74-41ff-9576-61d14d012c8b-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:41.539737 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:41.539706    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"
Sep 18 18:31:41.540145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:41.539708    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t" event={"ID":"394dc514-cf74-41ff-9576-61d14d012c8b","Type":"ContainerDied","Data":"de606045ec3da0b23251a8282c9db569f94eb9bc6f053f1620d769f76a243774"}
Sep 18 18:31:41.540145 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:41.539828    2550 scope.go:117] "RemoveContainer" containerID="1717390e405230514bef93ca228030a50da892bb5c1f980ef1e0689276920f38"
Sep 18 18:31:41.560523 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:41.560500    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:31:41.564146 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:41.564123    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-46c63c-6446498d6-zc69t"]
Sep 18 18:31:42.706223 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:42.706145    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="394dc514-cf74-41ff-9576-61d14d012c8b" path="/var/lib/kubelet/pods/394dc514-cf74-41ff-9576-61d14d012c8b/volumes"
Sep 18 18:31:43.581990 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:43.581926    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:31:47.502130 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502083    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"]
Sep 18 18:31:47.502564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502469    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="394dc514-cf74-41ff-9576-61d14d012c8b" containerName="payload-pre-processing"
Sep 18 18:31:47.502564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502482    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="394dc514-cf74-41ff-9576-61d14d012c8b" containerName="payload-pre-processing"
Sep 18 18:31:47.502564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502503    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" containerName="revoke-keys"
Sep 18 18:31:47.502564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502510    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" containerName="revoke-keys"
Sep 18 18:31:47.502564 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502564    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac2a71c-fdb1-4d98-980a-03dfd5faa5e9" containerName="revoke-keys"
Sep 18 18:31:47.502731 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.502575    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="394dc514-cf74-41ff-9576-61d14d012c8b" containerName="payload-pre-processing"
Sep 18 18:31:47.507026 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.507003    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:47.510156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.510123    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:31:47.510414 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.510363    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:31:47.512322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.512294    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"]
Sep 18 18:31:47.674958 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.674909    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5\" (UID: \"a4c8d084-b97d-44cd-abe7-2453183734e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:47.775550 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.775524    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5\" (UID: \"a4c8d084-b97d-44cd-abe7-2453183734e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:47.776172 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.776152    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5\" (UID: \"a4c8d084-b97d-44cd-abe7-2453183734e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:47.819669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:47.819643    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:48.087636 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:48.087614    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"]
Sep 18 18:31:48.089880 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:31:48.089853    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c8d084_b97d_44cd_abe7_2453183734e4.slice/crio-35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49 WatchSource:0}: Error finding container 35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49: Status 404 returned error can't find the container with id 35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49
Sep 18 18:31:48.581729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:48.581701    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5" event={"ID":"a4c8d084-b97d-44cd-abe7-2453183734e4","Type":"ContainerStarted","Data":"7ea0fec9530f65e633b7ab998cfaf0fd3caac8d012c005d4a4b8fad00e16029d"}
Sep 18 18:31:48.582100 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:48.581758    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5" event={"ID":"a4c8d084-b97d-44cd-abe7-2453183734e4","Type":"ContainerStarted","Data":"35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49"}
Sep 18 18:31:48.598557 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:48.598510    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5" podStartSLOduration=1.598497413 podStartE2EDuration="1.598497413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:31:48.596261411 +0000 UTC m=+1880.282890875" watchObservedRunningTime="2026-09-18 18:31:48.598497413 +0000 UTC m=+1880.285126865"
Sep 18 18:31:49.588871 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:49.588828    2550 generic.go:356] "Generic (PLEG): container finished" podID="a4c8d084-b97d-44cd-abe7-2453183734e4" containerID="7ea0fec9530f65e633b7ab998cfaf0fd3caac8d012c005d4a4b8fad00e16029d" exitCode=0
Sep 18 18:31:49.589279 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:49.588916    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5" event={"ID":"a4c8d084-b97d-44cd-abe7-2453183734e4","Type":"ContainerDied","Data":"7ea0fec9530f65e633b7ab998cfaf0fd3caac8d012c005d4a4b8fad00e16029d"}
Sep 18 18:31:50.730120 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:50.730092    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:50.898605 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:50.898543    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca\") pod \"a4c8d084-b97d-44cd-abe7-2453183734e4\" (UID: \"a4c8d084-b97d-44cd-abe7-2453183734e4\") "
Sep 18 18:31:50.898914 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:50.898892    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4c8d084-b97d-44cd-abe7-2453183734e4" (UID: "a4c8d084-b97d-44cd-abe7-2453183734e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:50.999912 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:50.999886    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4c8d084-b97d-44cd-abe7-2453183734e4-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:51.602219 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:51.602191    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"
Sep 18 18:31:51.602394 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:51.602228    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5" event={"ID":"a4c8d084-b97d-44cd-abe7-2453183734e4","Type":"ContainerDied","Data":"35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49"}
Sep 18 18:31:51.602394 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:51.602271    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35518a8a7de94aaedc797270fa2324d6d972e07d0231e5f19fb4f8e3076c6b49"
Sep 18 18:31:57.966127 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:57.966088    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"]
Sep 18 18:31:57.970136 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:57.970104    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-011541-29829270-tz5lf"]
Sep 18 18:31:57.996527 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:57.996503    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:31:57.996711 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:57.996689    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" podUID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" containerName="payload-pre-processing" containerID="cri-o://932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd" gracePeriod=30
Sep 18 18:31:58.184164 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.184133    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"]
Sep 18 18:31:58.187088 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.187060    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-011541-fs2h5"]
Sep 18 18:31:58.247353 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.247329    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:31:58.355375 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.355334    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume\") pod \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") "
Sep 18 18:31:58.355539 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.355440    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww6vc\" (UniqueName: \"kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc\") pod \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\" (UID: \"ca105863-19e3-4dd4-b16e-6e82e0fe8f42\") "
Sep 18 18:31:58.355693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.355668    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ca105863-19e3-4dd4-b16e-6e82e0fe8f42" (UID: "ca105863-19e3-4dd4-b16e-6e82e0fe8f42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:31:58.357530 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.357506    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc" (OuterVolumeSpecName: "kube-api-access-ww6vc") pod "ca105863-19e3-4dd4-b16e-6e82e0fe8f42" (UID: "ca105863-19e3-4dd4-b16e-6e82e0fe8f42"). InnerVolumeSpecName "kube-api-access-ww6vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:31:58.456709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.456681    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.456709 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.456707    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww6vc\" (UniqueName: \"kubernetes.io/projected/ca105863-19e3-4dd4-b16e-6e82e0fe8f42-kube-api-access-ww6vc\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:31:58.641624 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.641592    2550 generic.go:356] "Generic (PLEG): container finished" podID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" containerID="932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd" exitCode=0
Sep 18 18:31:58.641757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.641643    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" event={"ID":"ca105863-19e3-4dd4-b16e-6e82e0fe8f42","Type":"ContainerDied","Data":"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"}
Sep 18 18:31:58.641757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.641649    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"
Sep 18 18:31:58.641757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.641685    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h" event={"ID":"ca105863-19e3-4dd4-b16e-6e82e0fe8f42","Type":"ContainerDied","Data":"8b27230ff2200c3098be9b3a426e104071e218b2b10b5284374eceee457fa793"}
Sep 18 18:31:58.641757 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.641706    2550 scope.go:117] "RemoveContainer" containerID="932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"
Sep 18 18:31:58.651119 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.651102    2550 scope.go:117] "RemoveContainer" containerID="932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"
Sep 18 18:31:58.651403 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:31:58.651384    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd\": container with ID starting with 932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd not found: ID does not exist" containerID="932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"
Sep 18 18:31:58.651471 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.651412    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd"} err="failed to get container status \"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd\": rpc error: code = NotFound desc = could not find container \"932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd\": container with ID starting with 932cc8da631b0f931ca0a1485aa38ff8db60591e14b558ef26bd3e3fa4f492dd not found: ID does not exist"
Sep 18 18:31:58.661951 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.661920    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:31:58.666198 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.666174    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-011541-65954d6c5d-f2q2h"]
Sep 18 18:31:58.705595 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.705573    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c8d084-b97d-44cd-abe7-2453183734e4" path="/var/lib/kubelet/pods/a4c8d084-b97d-44cd-abe7-2453183734e4/volumes"
Sep 18 18:31:58.705858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.705843    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac99cc7e-f468-45b9-bcc2-c20161ab6a18" path="/var/lib/kubelet/pods/ac99cc7e-f468-45b9-bcc2-c20161ab6a18/volumes"
Sep 18 18:31:58.706165 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:31:58.706152    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" path="/var/lib/kubelet/pods/ca105863-19e3-4dd4-b16e-6e82e0fe8f42/volumes"
Sep 18 18:32:00.974960 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:00.974916    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:32:05.022033 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.021998    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"]
Sep 18 18:32:05.022419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022330    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c8d084-b97d-44cd-abe7-2453183734e4" containerName="revoke-keys"
Sep 18 18:32:05.022419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022341    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c8d084-b97d-44cd-abe7-2453183734e4" containerName="revoke-keys"
Sep 18 18:32:05.022419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022356    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" containerName="payload-pre-processing"
Sep 18 18:32:05.022419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022362    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" containerName="payload-pre-processing"
Sep 18 18:32:05.022419 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022419    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca105863-19e3-4dd4-b16e-6e82e0fe8f42" containerName="payload-pre-processing"
Sep 18 18:32:05.022582 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.022427    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c8d084-b97d-44cd-abe7-2453183734e4" containerName="revoke-keys"
Sep 18 18:32:05.026602 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.026582    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:05.029128 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.029104    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4ghvf\""
Sep 18 18:32:05.029260 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.029130    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:32:05.033954 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.033909    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"]
Sep 18 18:32:05.208915 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.208887    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf\" (UID: \"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:05.309416 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.309356    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf\" (UID: \"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:05.309953 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.309918    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf\" (UID: \"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:05.338470 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.338453    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:05.612776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.612706    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"]
Sep 18 18:32:05.615512 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:32:05.615484    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62dab507_e8ce_4ec0_8fa2_e5a2e72b4be2.slice/crio-0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8 WatchSource:0}: Error finding container 0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8: Status 404 returned error can't find the container with id 0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8
Sep 18 18:32:05.679293 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:05.679271    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf" event={"ID":"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2","Type":"ContainerStarted","Data":"0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8"}
Sep 18 18:32:06.686917 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:06.686839    2550 generic.go:356] "Generic (PLEG): container finished" podID="62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" containerID="99677071fa6f2e360be187f70c2906f273cde19d32b6ebed392a747b743b4766" exitCode=0
Sep 18 18:32:06.686917 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:06.686886    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf" event={"ID":"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2","Type":"ContainerDied","Data":"99677071fa6f2e360be187f70c2906f273cde19d32b6ebed392a747b743b4766"}
Sep 18 18:32:07.823624 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:07.823598    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:07.929957 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:07.929901    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca\") pod \"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2\" (UID: \"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2\") "
Sep 18 18:32:07.930233 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:07.930212    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" (UID: "62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:32:08.030567 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:08.030537    2550 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2-maas-api-service-ca\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:08.700302 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:08.700268    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"
Sep 18 18:32:08.705972 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:08.705915    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf" event={"ID":"62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2","Type":"ContainerDied","Data":"0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8"}
Sep 18 18:32:08.706117 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:08.705990    2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bec69dccd1616719e71739026230a6ad4b54f34c9b381e73d860e67f97d76d8"
Sep 18 18:32:15.539547 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.539511    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"]
Sep 18 18:32:15.542010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.541983    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-99602c-29829270-v2vfr"]
Sep 18 18:32:15.565508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.565480    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:32:15.565832 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.565803    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" podUID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" containerName="payload-processing" containerID="cri-o://e430b06be60adc94ef2ebb81a76e806af0fc63eacce6d806b42b83e34d416048" gracePeriod=30
Sep 18 18:32:15.740626 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.740578    2550 generic.go:356] "Generic (PLEG): container finished" podID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" containerID="e430b06be60adc94ef2ebb81a76e806af0fc63eacce6d806b42b83e34d416048" exitCode=0
Sep 18 18:32:15.740759 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.740659    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" event={"ID":"e9fd1729-6f5b-4fb4-9240-e513cabed3b6","Type":"ContainerDied","Data":"e430b06be60adc94ef2ebb81a76e806af0fc63eacce6d806b42b83e34d416048"}
Sep 18 18:32:15.748194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.748167    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"]
Sep 18 18:32:15.752106 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.752079    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-99602c-mftdf"]
Sep 18 18:32:15.817500 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.817479    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:32:15.985632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.985605    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nx64\" (UniqueName: \"kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64\") pod \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") "
Sep 18 18:32:15.985632 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.985639    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume\") pod \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\" (UID: \"e9fd1729-6f5b-4fb4-9240-e513cabed3b6\") "
Sep 18 18:32:15.986098 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.986071    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e9fd1729-6f5b-4fb4-9240-e513cabed3b6" (UID: "e9fd1729-6f5b-4fb4-9240-e513cabed3b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:32:15.987693 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:15.987667    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64" (OuterVolumeSpecName: "kube-api-access-2nx64") pod "e9fd1729-6f5b-4fb4-9240-e513cabed3b6" (UID: "e9fd1729-6f5b-4fb4-9240-e513cabed3b6"). InnerVolumeSpecName "kube-api-access-2nx64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:16.086841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.086778    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nx64\" (UniqueName: \"kubernetes.io/projected/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-kube-api-access-2nx64\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:16.086841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.086804    2550 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9fd1729-6f5b-4fb4-9240-e513cabed3b6-plugins-config-volume\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:16.706282 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.706249    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" path="/var/lib/kubelet/pods/62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2/volumes"
Sep 18 18:32:16.706648 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.706517    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d6a7ba-f895-46ce-a8a4-828b078f5fe8" path="/var/lib/kubelet/pods/86d6a7ba-f895-46ce-a8a4-828b078f5fe8/volumes"
Sep 18 18:32:16.747187 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.747161    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm" event={"ID":"e9fd1729-6f5b-4fb4-9240-e513cabed3b6","Type":"ContainerDied","Data":"b535c3fcb7121266f2885e6337a8f74a43ef6588f0d6992788c69b782ecf9351"}
Sep 18 18:32:16.747324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.747228    2550 scope.go:117] "RemoveContainer" containerID="e430b06be60adc94ef2ebb81a76e806af0fc63eacce6d806b42b83e34d416048"
Sep 18 18:32:16.747324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.747170    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"
Sep 18 18:32:16.763253 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.763232    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:32:16.769019 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:16.768998    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-99602c-7bf756d877-2jlzm"]
Sep 18 18:32:18.484189 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:18.484152    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:32:18.705800 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:18.705768    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" path="/var/lib/kubelet/pods/e9fd1729-6f5b-4fb4-9240-e513cabed3b6/volumes"
Sep 18 18:32:22.398809 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:22.398769    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:32:22.399224 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:22.399085    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy" containerID="cri-o://66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9" gracePeriod=30
Sep 18 18:32:26.486336 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:26.486292    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:32:27.656428 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.656399    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:32:27.671760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671704    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhq42\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.671760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671741    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.671905 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671782    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.671905 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671886    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.671992 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671955    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.671992 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.671979    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.672099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672025    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.672099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672032    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:32:27.672099 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672073    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.672258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672096    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:32:27.672258 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672131    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket\") pod \"05790c99-3826-4f5e-add1-e82e5a96f1dc\" (UID: \"05790c99-3826-4f5e-add1-e82e5a96f1dc\") "
Sep 18 18:32:27.672381 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672292    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:32:27.672481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672395    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 18:32:27.672481 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672468    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:32:27.672651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672417    2550 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-data\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.672651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672507    2550 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.672651 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.672518    2550 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-workload-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.674063 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.674025    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:27.674156 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.674112    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:32:27.674364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.674333    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42" (OuterVolumeSpecName: "kube-api-access-bhq42") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "kube-api-access-bhq42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:32:27.674364 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.674333    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05790c99-3826-4f5e-add1-e82e5a96f1dc" (UID: "05790c99-3826-4f5e-add1-e82e5a96f1dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 18:32:27.772806 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772783    2550 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-envoy\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.772806 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772804    2550 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-token\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.772970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772813    2550 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05790c99-3826-4f5e-add1-e82e5a96f1dc-istio-podinfo\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.772970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772821    2550 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05790c99-3826-4f5e-add1-e82e5a96f1dc-istiod-ca-cert\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.772970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772831    2550 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05790c99-3826-4f5e-add1-e82e5a96f1dc-credential-socket\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.772970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.772839    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhq42\" (UniqueName: \"kubernetes.io/projected/05790c99-3826-4f5e-add1-e82e5a96f1dc-kube-api-access-bhq42\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:32:27.809097 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.809073    2550 generic.go:356] "Generic (PLEG): container finished" podID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerID="66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9" exitCode=0
Sep 18 18:32:27.809209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.809128    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"
Sep 18 18:32:27.809209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.809149    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerDied","Data":"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"}
Sep 18 18:32:27.809209 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.809201    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr" event={"ID":"05790c99-3826-4f5e-add1-e82e5a96f1dc","Type":"ContainerDied","Data":"cd5c62402c4f13188655d5d6535ac74bac87e89d65476f4220ea83f93efc57da"}
Sep 18 18:32:27.809356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.809223    2550 scope.go:117] "RemoveContainer" containerID="66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"
Sep 18 18:32:27.820796 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.820779    2550 scope.go:117] "RemoveContainer" containerID="ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"
Sep 18 18:32:27.828697 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.828679    2550 scope.go:117] "RemoveContainer" containerID="66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"
Sep 18 18:32:27.828987 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:32:27.828956    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9\": container with ID starting with 66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9 not found: ID does not exist" containerID="66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"
Sep 18 18:32:27.829079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.828993    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9"} err="failed to get container status \"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9\": rpc error: code = NotFound desc = could not find container \"66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9\": container with ID starting with 66e640af6863f591e979e399d2185b39dfa95b5b20e13360f2140ec029045ea9 not found: ID does not exist"
Sep 18 18:32:27.829079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.829016    2550 scope.go:117] "RemoveContainer" containerID="ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"
Sep 18 18:32:27.829284 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:32:27.829266    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450\": container with ID starting with ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450 not found: ID does not exist" containerID="ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"
Sep 18 18:32:27.829326 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.829289    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450"} err="failed to get container status \"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450\": rpc error: code = NotFound desc = could not find container \"ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450\": container with ID starting with ca13716e9405d56510d109e9d07ace7e42386c62022ee7a6d452f3987badc450 not found: ID does not exist"
Sep 18 18:32:27.833264 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.833243    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:32:27.837292 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:27.837271    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-99602c-openshift-default-68db6f4458-9sslr"]
Sep 18 18:32:28.706305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:28.706264    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" path="/var/lib/kubelet/pods/05790c99-3826-4f5e-add1-e82e5a96f1dc/volumes"
Sep 18 18:32:31.151229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:31.151202    2550 scope.go:117] "RemoveContainer" containerID="92027a8857604dedf23e5ae98b231eddb22eedc6eebd1f985f62c64d497201fe"
Sep 18 18:32:31.159329 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:31.159300    2550 scope.go:117] "RemoveContainer" containerID="df3ba59e049f49428742b458ac27c18c004d2224d57dae608af154cbfb271641"
Sep 18 18:32:31.166607 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:32:31.166589    2550 scope.go:117] "RemoveContainer" containerID="6c3012940e21b92862697ed13da7f08c19e918a4ace7cf9b9fb3b96bd867bf7f"
Sep 18 18:33:31.208021 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:33:31.207994    2550 scope.go:117] "RemoveContainer" containerID="6eee4cbe605d7d078c37eb4697dddeb6ac6920b42ed752822ca724107cc93cd8"
Sep 18 18:33:48.676861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:33:48.676828    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:34:12.678955 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:34:12.678912    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:34:20.487403 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:34:20.487359    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:34:28.778152 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:34:28.778117    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:34:47.378030 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:34:47.377950    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:34:53.183347 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:34:53.183309    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:35:30.842328 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:35:30.842297    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:35:30.847858 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:35:30.847836    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fxxp8_297e4ba8-5383-4fc6-b55c-3d1b3d5f01d0/ovn-acl-logging/0.log"
Sep 18 18:36:23.674270 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:23.674185    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:36:29.883841 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:29.883806    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:36:31.242249 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:31.242218    2550 scope.go:117] "RemoveContainer" containerID="3e391bb2c68743dca83851e422bd2ca6bf3ed1aa7ec94e20a9482e90bf9cf959"
Sep 18 18:36:31.253067 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:31.253047    2550 scope.go:117] "RemoveContainer" containerID="54cf5a6bfbfdf5754878b78c1df658ae5347dc70fc6f05fb74825e36137cbb9f"
Sep 18 18:36:31.261889 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:31.261871    2550 scope.go:117] "RemoveContainer" containerID="f60abd9950e2243ef239cc2d5595467e25eb8e992836eab10aa6eaa728f9563f"
Sep 18 18:36:31.270128 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:31.270112    2550 scope.go:117] "RemoveContainer" containerID="8b51cc18acc783eab10578e1c01ec845045d8b16ec1b60695d236fa18b402854"
Sep 18 18:36:31.277457 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:31.277438    2550 scope.go:117] "RemoveContainer" containerID="70dedd3d89bf53884610db581007021cb3cf1413de3fd7f3225dd35a7b2fa336"
Sep 18 18:36:44.186878 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.186847    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187183    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187194    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187205    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" containerName="revoke-keys"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187210    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" containerName="revoke-keys"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187226    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" containerName="payload-processing"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187232    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" containerName="payload-processing"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187284    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="62dab507-e8ce-4ec0-8fa2-e5a2e72b4be2" containerName="revoke-keys"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187291    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187298    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9fd1729-6f5b-4fb4-9240-e513cabed3b6" containerName="payload-processing"
Sep 18 18:36:44.187391 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.187302    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:36:44.192110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.192092    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.194447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.194426    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-db251fad-kserve-self-signed-certs\""
Sep 18 18:36:44.201508 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.201485    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:36:44.212424 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212403    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.212518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212435    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.212518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212455    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.212518 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212474    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.212625 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212591    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8pfv\" (UniqueName: \"kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.212664 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.212642    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313498 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313473    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313541    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313566    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313590 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313586    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313602    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313729 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313633    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8pfv\" (UniqueName: \"kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313917 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313886    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.313917 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.313912    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.314091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.314019    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.315802 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.315777    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.316009 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.315994    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.320969 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.320932    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8pfv\" (UniqueName: \"kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv\") pod \"e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") " pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.502325 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.502262    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:44.773337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:44.773308    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:36:44.776971 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:36:44.776917    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53c09ee2_0eb7_48bf_8048_2dd1fcaf0081.slice/crio-3bb1c91176f59ceed020e802ad8c697d3cd8900382a3dacdfeb04293cbd33f8a WatchSource:0}: Error finding container 3bb1c91176f59ceed020e802ad8c697d3cd8900382a3dacdfeb04293cbd33f8a: Status 404 returned error can't find the container with id 3bb1c91176f59ceed020e802ad8c697d3cd8900382a3dacdfeb04293cbd33f8a
Sep 18 18:36:45.088350 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:45.088272    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" event={"ID":"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081","Type":"ContainerStarted","Data":"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"}
Sep 18 18:36:45.088350 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:45.088311    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" event={"ID":"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081","Type":"ContainerStarted","Data":"3bb1c91176f59ceed020e802ad8c697d3cd8900382a3dacdfeb04293cbd33f8a"}
Sep 18 18:36:45.106484 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:45.106437    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" podStartSLOduration=1.106423333 podStartE2EDuration="1.106423333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:36:45.104973049 +0000 UTC m=+2176.791602505" watchObservedRunningTime="2026-09-18 18:36:45.106423333 +0000 UTC m=+2176.793052784"
Sep 18 18:36:46.683151 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:46.683114    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:36:54.503296 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:54.503258    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:54.503296 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:54.503305    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:54.516139 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:54.516107    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:36:55.145869 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:36:55.145844    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:37:04.182669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.182635    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:04.183295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.182969    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:37:04.183295 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.182985    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="05790c99-3826-4f5e-add1-e82e5a96f1dc" containerName="istio-proxy"
Sep 18 18:37:04.186020 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.185998    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.188766 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.188743    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-db251fad-kserve-self-signed-certs\""
Sep 18 18:37:04.195330 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.195307    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:04.286431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286391    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.286431 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286435    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.286645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286454    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.286645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286507    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.286645 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286562    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jjq\" (UniqueName: \"kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.286749 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.286643    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387219 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387182    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387219 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387225    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387243    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387264    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387293    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jjq\" (UniqueName: \"kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387423 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387364    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387663 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387638    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387727 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387668    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.387818 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.387723    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.389650 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.389616    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.389774 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.389755    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.395271 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.395245    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jjq\" (UniqueName: \"kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq\") pod \"e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") " pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.495967 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.495847    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:04.773896 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:04.773870    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:04.776873 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:37:04.776835    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56cb5840_fa0c_4725_8f87_230062ee7486.slice/crio-6944afaafd924df0b69950fd825b654784cf7d1156c021a361c5458000457596 WatchSource:0}: Error finding container 6944afaafd924df0b69950fd825b654784cf7d1156c021a361c5458000457596: Status 404 returned error can't find the container with id 6944afaafd924df0b69950fd825b654784cf7d1156c021a361c5458000457596
Sep 18 18:37:05.191125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:05.191016    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" event={"ID":"56cb5840-fa0c-4725-8f87-230062ee7486","Type":"ContainerStarted","Data":"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"}
Sep 18 18:37:05.191125 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:05.191063    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" event={"ID":"56cb5840-fa0c-4725-8f87-230062ee7486","Type":"ContainerStarted","Data":"6944afaafd924df0b69950fd825b654784cf7d1156c021a361c5458000457596"}
Sep 18 18:37:05.208577 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:05.208522    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" podStartSLOduration=1.208505448 podStartE2EDuration="1.208505448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:37:05.207063563 +0000 UTC m=+2196.893693045" watchObservedRunningTime="2026-09-18 18:37:05.208505448 +0000 UTC m=+2196.895134899"
Sep 18 18:37:07.098171 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:07.098127    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:14.496910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:14.496868    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:14.496910 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:14.496916    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:14.510726 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:14.510696    2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:15.250424 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:15.250387    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:18.443437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:18.443404    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:37:19.486767 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:19.486730    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:21.676337 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:21.676301    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:29.086970 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:29.086923    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:31.306996 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:31.306967    2550 scope.go:117] "RemoveContainer" containerID="502caf2c389085411b57bd64dbf98cd18b42bda5a997cebc4096f5a57059902a"
Sep 18 18:37:33.462679 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.462643    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" podUID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" containerName="main" containerID="cri-o://6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5" gracePeriod=45
Sep 18 18:37:33.711760 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.711733    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:37:33.744887 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.744817    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.744896    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.744926    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.744979    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.744996    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745053 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745020    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8pfv\" (UniqueName: \"kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv\") pod \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\" (UID: \"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081\") "
Sep 18 18:37:33.745324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745107    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:33.745324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745226    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home" (OuterVolumeSpecName: "home") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:33.745324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745303    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache" (OuterVolumeSpecName: "model-cache") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:33.745494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745426    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:33.745494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745445    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:33.745494 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.745460    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:33.747520 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.747477    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:37:33.747520 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.747516    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm" (OuterVolumeSpecName: "dshm") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:33.747695 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.747569    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv" (OuterVolumeSpecName: "kube-api-access-r8pfv") pod "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" (UID: "53c09ee2-0eb7-48bf-8048-2dd1fcaf0081"). InnerVolumeSpecName "kube-api-access-r8pfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:37:33.846305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.846274    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:33.846305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.846300    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:33.846305 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:33.846311    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8pfv\" (UniqueName: \"kubernetes.io/projected/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081-kube-api-access-r8pfv\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.335844 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.335809    2550 generic.go:356] "Generic (PLEG): container finished" podID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" containerID="6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5" exitCode=0
Sep 18 18:37:34.336045 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.335851    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"
Sep 18 18:37:34.336045 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.335857    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" event={"ID":"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081","Type":"ContainerDied","Data":"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"}
Sep 18 18:37:34.336045 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.335964    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm" event={"ID":"53c09ee2-0eb7-48bf-8048-2dd1fcaf0081","Type":"ContainerDied","Data":"3bb1c91176f59ceed020e802ad8c697d3cd8900382a3dacdfeb04293cbd33f8a"}
Sep 18 18:37:34.336045 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.335981    2550 scope.go:117] "RemoveContainer" containerID="6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"
Sep 18 18:37:34.345308 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.345287    2550 scope.go:117] "RemoveContainer" containerID="6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"
Sep 18 18:37:34.345565 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:37:34.345537    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5\": container with ID starting with 6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5 not found: ID does not exist" containerID="6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"
Sep 18 18:37:34.345613 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.345578    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5"} err="failed to get container status \"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5\": rpc error: code = NotFound desc = could not find container \"6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5\": container with ID starting with 6d54426f8bb2ef3725330c3b548eb8605e13c09925c922153f1e407695e09ff5 not found: ID does not exist"
Sep 18 18:37:34.355770 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.355749    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:37:34.359901 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.359879    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-db251fad-kserve-66845ff74c-pn6lm"]
Sep 18 18:37:34.393839 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.393819    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:34.507495 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.507452    2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" podUID="56cb5840-fa0c-4725-8f87-230062ee7486" containerName="main" containerID="cri-o://2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9" gracePeriod=45
Sep 18 18:37:34.706532 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.706499    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" path="/var/lib/kubelet/pods/53c09ee2-0eb7-48bf-8048-2dd1fcaf0081/volumes"
Sep 18 18:37:34.753898 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.753874    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:34.853920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.853886    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.853920 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.853924    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.854194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.853970    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9jjq\" (UniqueName: \"kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.854194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.853988    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.854194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854017    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.854194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854106    2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir\") pod \"56cb5840-fa0c-4725-8f87-230062ee7486\" (UID: \"56cb5840-fa0c-4725-8f87-230062ee7486\") "
Sep 18 18:37:34.854194 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854153    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home" (OuterVolumeSpecName: "home") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:34.854455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854276    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache" (OuterVolumeSpecName: "model-cache") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:34.854455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854383    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:34.854455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854392    2550 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-home\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.854455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.854410    2550 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-model-cache\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.856126 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.856100    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm" (OuterVolumeSpecName: "dshm") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 18:37:34.856231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.856175    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq" (OuterVolumeSpecName: "kube-api-access-r9jjq") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "kube-api-access-r9jjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 18:37:34.856231 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.856200    2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "56cb5840-fa0c-4725-8f87-230062ee7486" (UID: "56cb5840-fa0c-4725-8f87-230062ee7486"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 18:37:34.955492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.955406    2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9jjq\" (UniqueName: \"kubernetes.io/projected/56cb5840-fa0c-4725-8f87-230062ee7486-kube-api-access-r9jjq\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.955492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.955434    2550 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56cb5840-fa0c-4725-8f87-230062ee7486-tls-certs\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.955492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.955446    2550 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-dshm\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:34.955492 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:34.955457    2550 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56cb5840-fa0c-4725-8f87-230062ee7486-tmp-dir\") on node \"ip-10-0-129-128.ec2.internal\" DevicePath \"\""
Sep 18 18:37:35.342454 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.342415    2550 generic.go:356] "Generic (PLEG): container finished" podID="56cb5840-fa0c-4725-8f87-230062ee7486" containerID="2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9" exitCode=0
Sep 18 18:37:35.342735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.342464    2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"
Sep 18 18:37:35.342735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.342506    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" event={"ID":"56cb5840-fa0c-4725-8f87-230062ee7486","Type":"ContainerDied","Data":"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"}
Sep 18 18:37:35.342735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.342546    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c" event={"ID":"56cb5840-fa0c-4725-8f87-230062ee7486","Type":"ContainerDied","Data":"6944afaafd924df0b69950fd825b654784cf7d1156c021a361c5458000457596"}
Sep 18 18:37:35.342735 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.342567    2550 scope.go:117] "RemoveContainer" containerID="2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"
Sep 18 18:37:35.351461 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.351445    2550 scope.go:117] "RemoveContainer" containerID="2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"
Sep 18 18:37:35.351714 ip-10-0-129-128 kubenswrapper[2550]: E0918 18:37:35.351697    2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9\": container with ID starting with 2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9 not found: ID does not exist" containerID="2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"
Sep 18 18:37:35.351765 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.351720    2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9"} err="failed to get container status \"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9\": rpc error: code = NotFound desc = could not find container \"2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9\": container with ID starting with 2e738ab05c31c43b0505478af7f0fd950117b660fd8af5a73c3dd67529493fc9 not found: ID does not exist"
Sep 18 18:37:35.362121 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.362099    2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:35.366497 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:35.366476    2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-db251fad-kserve-bbfff7679-6t92c"]
Sep 18 18:37:36.705822 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:36.705790    2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56cb5840-fa0c-4725-8f87-230062ee7486" path="/var/lib/kubelet/pods/56cb5840-fa0c-4725-8f87-230062ee7486/volumes"
Sep 18 18:37:40.782272 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:40.782238    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:41.887577 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887540    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"]
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887854    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56cb5840-fa0c-4725-8f87-230062ee7486" containerName="main"
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887867    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="56cb5840-fa0c-4725-8f87-230062ee7486" containerName="main"
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887890    2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" containerName="main"
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887895    2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" containerName="main"
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887962    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="56cb5840-fa0c-4725-8f87-230062ee7486" containerName="main"
Sep 18 18:37:41.887983 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.887971    2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c09ee2-0eb7-48bf-8048-2dd1fcaf0081" containerName="main"
Sep 18 18:37:41.892217 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.892199    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:41.894571 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.894546    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 18:37:41.894700 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.894601    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 18:37:41.895374 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.895358    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 18:37:41.895455 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.895363    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-jkbfn\""
Sep 18 18:37:41.895763 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.895746    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 18:37:41.900386 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:41.900362    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"]
Sep 18 18:37:42.021751 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.021670    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-metrics-certs\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.021919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.021748    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11380a12-80ae-4500-ae55-56facacb5e51-openshift-service-ca\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.021919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.021781    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-maas-api-tls\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.021919 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.021816    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl98j\" (UniqueName: \"kubernetes.io/projected/11380a12-80ae-4500-ae55-56facacb5e51-kube-api-access-fl98j\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.122319 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.122282    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl98j\" (UniqueName: \"kubernetes.io/projected/11380a12-80ae-4500-ae55-56facacb5e51-kube-api-access-fl98j\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.122465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.122326    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-metrics-certs\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.122465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.122372    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11380a12-80ae-4500-ae55-56facacb5e51-openshift-service-ca\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.122465 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.122393    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-maas-api-tls\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.123135 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.123109    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11380a12-80ae-4500-ae55-56facacb5e51-openshift-service-ca\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.124861 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.124838    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-metrics-certs\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.124989 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.124927    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11380a12-80ae-4500-ae55-56facacb5e51-maas-api-tls\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.129828 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.129805    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl98j\" (UniqueName: \"kubernetes.io/projected/11380a12-80ae-4500-ae55-56facacb5e51-kube-api-access-fl98j\") pod \"maas-api-86d9ddc875-4q8fr\" (UID: \"11380a12-80ae-4500-ae55-56facacb5e51\") " pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.204462 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.204366    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:42.478840 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.478701    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"]
Sep 18 18:37:42.481816 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:37:42.481785    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11380a12_80ae_4500_ae55_56facacb5e51.slice/crio-3c4598cb366b27b615481953ad59a4ef3be9ed5f15cd8434e66e93508458f10f WatchSource:0}: Error finding container 3c4598cb366b27b615481953ad59a4ef3be9ed5f15cd8434e66e93508458f10f: Status 404 returned error can't find the container with id 3c4598cb366b27b615481953ad59a4ef3be9ed5f15cd8434e66e93508458f10f
Sep 18 18:37:42.483467 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:42.483447    2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 18:37:43.382780 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:43.382723    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr" event={"ID":"11380a12-80ae-4500-ae55-56facacb5e51","Type":"ContainerStarted","Data":"3c4598cb366b27b615481953ad59a4ef3be9ed5f15cd8434e66e93508458f10f"}
Sep 18 18:37:45.395560 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:45.395523    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr" event={"ID":"11380a12-80ae-4500-ae55-56facacb5e51","Type":"ContainerStarted","Data":"5b1e22b7956f46842be81c27c4623570e543ed5a5a10fe28c6fb60c3bdd1bcae"}
Sep 18 18:37:45.395995 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:45.395669    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:45.411825 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:45.411769    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr" podStartSLOduration=1.9308415060000002 podStartE2EDuration="4.411750165s" totalImagesPullingTime="2.480908659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:37:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:37:45.410767151 +0000 UTC m=+2237.097396602" watchObservedRunningTime="2026-09-18 18:37:45.411750165 +0000 UTC m=+2237.098379618"
Sep 18 18:37:47.980776 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:47.980743    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:37:51.405322 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:51.405292    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86d9ddc875-4q8fr"
Sep 18 18:37:53.675327 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:37:53.675287    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:38:01.082153 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:01.082115    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-nrhnf"]
Sep 18 18:38:04.248044 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.248006    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8m5v/must-gather-dfwpv"]
Sep 18 18:38:04.254110 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.254078    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.257091 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.257063    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8m5v\"/\"kube-root-ca.crt\""
Sep 18 18:38:04.257234 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.257099    2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-x8m5v\"/\"openshift-service-ca.crt\""
Sep 18 18:38:04.257284 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.257257    2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-x8m5v\"/\"default-dockercfg-qmjsc\""
Sep 18 18:38:04.261324 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.261298    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/must-gather-dfwpv"]
Sep 18 18:38:04.414468 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.414437    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwzhz\" (UniqueName: \"kubernetes.io/projected/8246498e-beab-404d-9896-3f9e5997d5f8-kube-api-access-xwzhz\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.414653 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.414483    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8246498e-beab-404d-9896-3f9e5997d5f8-must-gather-output\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.515635 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.515611    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwzhz\" (UniqueName: \"kubernetes.io/projected/8246498e-beab-404d-9896-3f9e5997d5f8-kube-api-access-xwzhz\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.515792 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.515647    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8246498e-beab-404d-9896-3f9e5997d5f8-must-gather-output\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.516010 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.515993    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8246498e-beab-404d-9896-3f9e5997d5f8-must-gather-output\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.524334 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.524316    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwzhz\" (UniqueName: \"kubernetes.io/projected/8246498e-beab-404d-9896-3f9e5997d5f8-kube-api-access-xwzhz\") pod \"must-gather-dfwpv\" (UID: \"8246498e-beab-404d-9896-3f9e5997d5f8\") " pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.566447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.566422    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/must-gather-dfwpv"
Sep 18 18:38:04.838450 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:04.838422    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/must-gather-dfwpv"]
Sep 18 18:38:04.840981 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:38:04.840959    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8246498e_beab_404d_9896_3f9e5997d5f8.slice/crio-d5a9963e16a85a60a9a769349e0116799c4772d780c326ac8f3350bdac698b16 WatchSource:0}: Error finding container d5a9963e16a85a60a9a769349e0116799c4772d780c326ac8f3350bdac698b16: Status 404 returned error can't find the container with id d5a9963e16a85a60a9a769349e0116799c4772d780c326ac8f3350bdac698b16
Sep 18 18:38:05.489773 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:05.489736    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/must-gather-dfwpv" event={"ID":"8246498e-beab-404d-9896-3f9e5997d5f8","Type":"ContainerStarted","Data":"d5a9963e16a85a60a9a769349e0116799c4772d780c326ac8f3350bdac698b16"}
Sep 18 18:38:06.499174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:06.499133    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/must-gather-dfwpv" event={"ID":"8246498e-beab-404d-9896-3f9e5997d5f8","Type":"ContainerStarted","Data":"f694c08a2fb2ccce969d0cfb26d61842efd7984579155b8c22ef2887562429eb"}
Sep 18 18:38:06.499174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:06.499178    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/must-gather-dfwpv" event={"ID":"8246498e-beab-404d-9896-3f9e5997d5f8","Type":"ContainerStarted","Data":"1194cc1f74eaa72a3cdf9f9082a5cf1a3b6ea6d41a98bc8fd9b6a002be59f2d8"}
Sep 18 18:38:07.304794 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:07.304761    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qchwl_581c7670-302a-4aaf-b37a-e594cda3d5cb/global-pull-secret-syncer/0.log"
Sep 18 18:38:07.463340 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:07.463309    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t9tq7_5f1b9143-08b9-41d7-a2f7-7569c371a3df/konnectivity-agent/0.log"
Sep 18 18:38:07.482556 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:07.482519    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-128.ec2.internal_5dcfdef0994e6b71a1bdabdbc7dab2b8/haproxy/0.log"
Sep 18 18:38:12.178963 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:12.178008    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-b6f99bdff-xvbfq_d9cf2dbb-81fd-4da3-afc7-b40bf0208287/authorino/0.log"
Sep 18 18:38:12.263306 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:12.263271    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-xzqtv_b0920464-7501-438b-a967-6b70a9cf9106/kuadrant-console-plugin/0.log"
Sep 18 18:38:12.327880 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:12.327840    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-nrhnf_6725978e-bd86-406b-aa97-fd2d5d66ba32/limitador/0.log"
Sep 18 18:38:13.738574 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.738537    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/alertmanager/0.log"
Sep 18 18:38:13.759375 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.759326    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/config-reloader/0.log"
Sep 18 18:38:13.780366 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.780321    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/kube-rbac-proxy-web/0.log"
Sep 18 18:38:13.800803 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.800777    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/kube-rbac-proxy/0.log"
Sep 18 18:38:13.819684 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.819660    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/kube-rbac-proxy-metric/0.log"
Sep 18 18:38:13.839265 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.839232    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/prom-label-proxy/0.log"
Sep 18 18:38:13.860799 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:13.860746    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_eac7fbe7-acc3-4395-98c6-f7dda09db45a/init-config-reloader/0.log"
Sep 18 18:38:14.177570 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.177505    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh7fp_af3a07a2-265b-468a-8612-e40c5471451f/node-exporter/0.log"
Sep 18 18:38:14.196650 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.196623    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh7fp_af3a07a2-265b-468a-8612-e40c5471451f/kube-rbac-proxy/0.log"
Sep 18 18:38:14.221342 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.221308    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xh7fp_af3a07a2-265b-468a-8612-e40c5471451f/init-textfile/0.log"
Sep 18 18:38:14.248895 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.248864    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-d2v4l_bb6417c9-ff62-4855-9648-4e2056414030/kube-rbac-proxy-main/0.log"
Sep 18 18:38:14.276056 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.276024    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-d2v4l_bb6417c9-ff62-4855-9648-4e2056414030/kube-rbac-proxy-self/0.log"
Sep 18 18:38:14.300875 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.300844    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-d2v4l_bb6417c9-ff62-4855-9648-4e2056414030/openshift-state-metrics/0.log"
Sep 18 18:38:14.631600 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:14.631506    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-bbvws_b1ab9906-1291-4351-afa2-84b9764270ca/prometheus-operator-admission-webhook/0.log"
Sep 18 18:38:17.359311 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.359224    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8m5v/must-gather-dfwpv" podStartSLOduration=12.547837424 podStartE2EDuration="13.359201372s" totalImagesPullingTime="811.363948ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 18:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:38:06.515163115 +0000 UTC m=+2258.201792568" watchObservedRunningTime="2026-09-18 18:38:17.359201372 +0000 UTC m=+2269.045830825"
Sep 18 18:38:17.360721 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.360683    2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"]
Sep 18 18:38:17.364367 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.364342    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.373149 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.373111    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"]
Sep 18 18:38:17.453166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.451073    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-lib-modules\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.453166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.451156    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-sys\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.453166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.451201    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-podres\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.453166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.451243    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtd5l\" (UniqueName: \"kubernetes.io/projected/9073e465-313a-4ff2-83b4-aa7e31f053f9-kube-api-access-mtd5l\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.453166 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.451271    2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-proc\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.552456 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.552415    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-sys\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.552859 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.552836    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-podres\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.553199 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.553181    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtd5l\" (UniqueName: \"kubernetes.io/projected/9073e465-313a-4ff2-83b4-aa7e31f053f9-kube-api-access-mtd5l\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.553669 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.553651    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-proc\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.553918 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.553904    2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-lib-modules\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.554226 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.553817    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-proc\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.554341 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.552759    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-sys\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.554437 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.554125    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-lib-modules\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.554531 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.553102    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9073e465-313a-4ff2-83b4-aa7e31f053f9-podres\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.561446 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.561426    2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtd5l\" (UniqueName: \"kubernetes.io/projected/9073e465-313a-4ff2-83b4-aa7e31f053f9-kube-api-access-mtd5l\") pod \"perf-node-gather-daemonset-sj5jt\" (UID: \"9073e465-313a-4ff2-83b4-aa7e31f053f9\") " pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.661021 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.660918    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-fbhkj_73ac416f-3856-4fdb-a9a6-fb93aa044abd/download-server/0.log"
Sep 18 18:38:17.684592 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.684558    2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:17.989788 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:17.989751    2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"]
Sep 18 18:38:17.995973 ip-10-0-129-128 kubenswrapper[2550]: W0918 18:38:17.995917    2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9073e465_313a_4ff2_83b4_aa7e31f053f9.slice/crio-1d4aa9f0deba244ee9767c270460d0baa6073de58bc3016a10f55ec5b32f7c52 WatchSource:0}: Error finding container 1d4aa9f0deba244ee9767c270460d0baa6073de58bc3016a10f55ec5b32f7c52: Status 404 returned error can't find the container with id 1d4aa9f0deba244ee9767c270460d0baa6073de58bc3016a10f55ec5b32f7c52
Sep 18 18:38:18.620705 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:18.620660    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt" event={"ID":"9073e465-313a-4ff2-83b4-aa7e31f053f9","Type":"ContainerStarted","Data":"fefa3802f044c4a53461938a536ab541c5149c90fa4a664f95bd637f83e8c16d"}
Sep 18 18:38:18.621186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:18.620729    2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt" event={"ID":"9073e465-313a-4ff2-83b4-aa7e31f053f9","Type":"ContainerStarted","Data":"1d4aa9f0deba244ee9767c270460d0baa6073de58bc3016a10f55ec5b32f7c52"}
Sep 18 18:38:18.621186 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:18.621040    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:18.637358 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:18.637297    2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt" podStartSLOduration=1.637274524 podStartE2EDuration="1.637274524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 18:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 18:38:18.635753941 +0000 UTC m=+2270.322383416" watchObservedRunningTime="2026-09-18 18:38:18.637274524 +0000 UTC m=+2270.323903978"
Sep 18 18:38:19.613287 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:19.613256    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bnwbx_b84f3fac-d7dc-4b4b-ba72-e27f520a0713/dns/0.log"
Sep 18 18:38:19.634137 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:19.634111    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bnwbx_b84f3fac-d7dc-4b4b-ba72-e27f520a0713/kube-rbac-proxy/0.log"
Sep 18 18:38:19.743065 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:19.743040    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6ndzh_783025bc-6935-4e23-9f5e-13bf0a719c7d/dns-node-resolver/0.log"
Sep 18 18:38:20.305682 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:20.305652    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c4dd8f4c-l92vt_c01cf412-f69a-47b2-a5d2-b4c2f7c33b9e/registry/0.log"
Sep 18 18:38:20.370016 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:20.369987    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rvfwt_40a87f8b-11fa-4221-afaf-f876077d444a/node-ca/0.log"
Sep 18 18:38:21.663356 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:21.663328    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-ssznq_11277bc3-e440-41ee-bf9e-277f4a46849d/kube-auth-proxy/0.log"
Sep 18 18:38:22.663886 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:22.663848    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v52xc_1e9ca0b1-e151-4043-bb0b-0c7344d8e10a/serve-healthcheck-canary/0.log"
Sep 18 18:38:23.124957 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:23.123348    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cgncr_6033eb70-c797-402e-8355-bb174ecb6106/kube-rbac-proxy/0.log"
Sep 18 18:38:23.146229 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:23.146199    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cgncr_6033eb70-c797-402e-8355-bb174ecb6106/exporter/0.log"
Sep 18 18:38:23.166949 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:23.166915    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cgncr_6033eb70-c797-402e-8355-bb174ecb6106/extractor/0.log"
Sep 18 18:38:24.634903 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:24.634873    2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-x8m5v/perf-node-gather-daemonset-sj5jt"
Sep 18 18:38:26.013081 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:26.013055    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57cbb766c6-cxxrk_b0a47825-8f15-4d6b-87bf-abc34350a349/manager/0.log"
Sep 18 18:38:26.032405 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:26.032381    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57cbb766c6-cxxrk_b0a47825-8f15-4d6b-87bf-abc34350a349/copy-manifests/0.log"
Sep 18 18:38:27.609850 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:27.609803    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d5f876c6c-zndc7_4fdb506b-6101-4677-b0d3-516804a6bdbc/manager/0.log"
Sep 18 18:38:28.085079 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:28.085021    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-ssznq_11277bc3-e440-41ee-bf9e-277f4a46849d/kube-auth-proxy/0.log"
Sep 18 18:38:30.028251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:30.028186    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9fbf455f6-ssznq_11277bc3-e440-41ee-bf9e-277f4a46849d/kube-auth-proxy/0.log"
Sep 18 18:38:31.330666 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:31.330633    2550 scope.go:117] "RemoveContainer" containerID="99677071fa6f2e360be187f70c2906f273cde19d32b6ebed392a747b743b4766"
Sep 18 18:38:31.346480 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:31.346446    2550 scope.go:117] "RemoveContainer" containerID="7ea0fec9530f65e633b7ab998cfaf0fd3caac8d012c005d4a4b8fad00e16029d"
Sep 18 18:38:32.581489 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:32.581463    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-l7pg6_bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35/migrator/0.log"
Sep 18 18:38:32.600174 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:32.600145    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-l7pg6_bf0edd7c-0b67-46ff-b5b3-5f7880ca1c35/graceful-termination/0.log"
Sep 18 18:38:33.915923 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:33.915891    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/kube-multus-additional-cni-plugins/0.log"
Sep 18 18:38:33.934043 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:33.934014    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/egress-router-binary-copy/0.log"
Sep 18 18:38:33.952114 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:33.952094    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/cni-plugins/0.log"
Sep 18 18:38:33.971543 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:33.971524    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/bond-cni-plugin/0.log"
Sep 18 18:38:33.989927 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:33.989903    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/routeoverride-cni/0.log"
Sep 18 18:38:34.008346 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:34.008327    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/whereabouts-cni-bincopy/0.log"
Sep 18 18:38:34.027004 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:34.026986    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2svdh_9649b91e-5c5d-4b53-8685-05e25dfc10d6/whereabouts-cni/0.log"
Sep 18 18:38:34.356251 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:34.356222    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mqhhp_28641cac-4605-4c9f-af83-a23a3d2c09a8/kube-multus/0.log"
Sep 18 18:38:34.432179 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:34.432111    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7bt9q_68cc169a-4dc4-4458-85f9-46ffb8565d59/network-metrics-daemon/0.log"
Sep 18 18:38:34.450447 ip-10-0-129-128 kubenswrapper[2550]: I0918 18:38:34.450426    2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7bt9q_68cc169a-4dc4-4458-85f9-46ffb8565d59/kube-rbac-proxy/0.log"