Sep 10 11:14:25.086593 ip-10-0-132-86 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 11:14:25.465363 ip-10-0-132-86 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:14:25.465363 ip-10-0-132-86 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 11:14:25.465363 ip-10-0-132-86 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:14:25.465363 ip-10-0-132-86 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 11:14:25.465363 ip-10-0-132-86 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 11:14:25.466016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.465454    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 11:14:25.469143 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469126    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.469143 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469142    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.469143 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469145    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469148    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469152    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469155    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469158    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469161    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469163    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469167    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469172    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469175    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469178    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469181    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469183    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469186    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469188    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469192    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469196    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469199    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469202    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.469229 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469205    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469208    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469210    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469213    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469216    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469219    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469221    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469224    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469227    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469230    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469232    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469235    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469238    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469241    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469244    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469247    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469250    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469252    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469255    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.469704 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469257    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469260    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469263    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469266    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469269    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469271    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469274    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469276    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469279    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469281    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469284    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469286    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469289    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469291    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469294    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469308    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469311    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469314    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469317    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469319    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469322    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.470214 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469325    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469327    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469330    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469332    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469335    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469338    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469340    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469343    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469345    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469348    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469357    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469360    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469362    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469365    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469367    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469371    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469375    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469379    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469381    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469384    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.470737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469387    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469389    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469392    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469395    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469397    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469400    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469404    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469407    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469410    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469412    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469415    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469417    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469420    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469423    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469425    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469428    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469436    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.469438    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470231    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470238    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.471221 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470242    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470245    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470249    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470257    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470260    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470263    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470266    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470269    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470271    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470274    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470276    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470280    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470282    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470285    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470287    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470290    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470293    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470295    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470316    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470321    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.471714 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470325    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470329    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470332    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470335    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470337    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470340    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470342    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470345    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470347    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470350    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470352    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470355    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470359    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470362    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470365    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470367    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470372    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470374    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470377    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.472211 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470380    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470382    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470386    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470389    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470393    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470396    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470399    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470402    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470405    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470408    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470411    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470415    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470418    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470421    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470424    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470427    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470430    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470432    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470435    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.472681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470438    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470440    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470443    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470445    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470448    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470451    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470453    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470456    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470458    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470460    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470463    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470466    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470469    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470471    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470473    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470476    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470478    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470481    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470484    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470486    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.473154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470489    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470491    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470494    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470498    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470502    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470504    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470507    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470510    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470512    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470514    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470517    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470520    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470522    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470525    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470528    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470531    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470533    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470536    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470538    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.473708 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470911    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470916    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470918    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470921    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470925    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470927    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470930    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470933    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470936    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470940    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470943    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470946    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470949    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470951    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470955    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470959    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470962    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470965    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470968    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.474215 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470971    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470973    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470976    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470979    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470981    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470984    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470987    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470990    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470992    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470995    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.470997    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471000    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471003    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471006    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471008    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471011    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471014    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471016    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471019    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471022    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.474692 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471024    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471027    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471031    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471033    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471036    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471038    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471041    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471043    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471046    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471048    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471051    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471053    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471057    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471061    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471063    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471066    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471069    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471072    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471074    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.475171 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471077    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471079    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471082    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471084    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471087    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471089    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471092    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471095    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471097    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471100    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471102    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471105    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471108    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471111    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471113    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471117    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471120    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471122    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471125    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471127    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.475678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471130    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471133    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471137    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471140    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471143    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471146    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471148    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471151    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471153    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471156    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471159    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471161    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471164    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471166    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471168    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471171    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471174    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471177    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471179    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.476170 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471181    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471184    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471261    2585 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471293    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471316    2585 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471320    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471325    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471328    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471333    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471338    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471341    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471344    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471347    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471351    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471354    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471357    2585 flags.go:64] FLAG: --cgroup-root=""
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471362    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471365    2585 flags.go:64] FLAG: --client-ca-file=""
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471368    2585 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471371    2585 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471375    2585 flags.go:64] FLAG: --cluster-domain=""
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471378    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471381    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471384    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 11:14:25.476630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471387    2585 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471391    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471393    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471396    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471400    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471403    2585 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471406    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471408    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471411    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471414    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471421    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471423    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471426    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471429    2585 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471432    2585 flags.go:64] FLAG: --enable-server="true"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471435    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471441    2585 flags.go:64] FLAG: --event-burst="100"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471444    2585 flags.go:64] FLAG: --event-qps="50"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471447    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471450    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471453    2585 flags.go:64] FLAG: --eviction-hard=""
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471457    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471460    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471463    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471467    2585 flags.go:64] FLAG: --eviction-soft=""
Sep 10 11:14:25.477241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471470    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471472    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471476    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471479    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471481    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471484    2585 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471487    2585 flags.go:64] FLAG: --feature-gates=""
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471491    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471494    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471496    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471499    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471502    2585 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471505    2585 flags.go:64] FLAG: --help="false"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471508    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471511    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471514    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471517    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471520    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471523    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471526    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471529    2585 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471532    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471534    2585 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 11:14:25.477863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471538    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471541    2585 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471544    2585 flags.go:64] FLAG: --kube-reserved=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471547    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471550    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471553    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471556    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471559    2585 flags.go:64] FLAG: --lock-file=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471563    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471565    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471568    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471573    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471576    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471579    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471582    2585 flags.go:64] FLAG: --logging-format="text"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471585    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471588    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471591    2585 flags.go:64] FLAG: --manifest-url=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471593    2585 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471598    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471601    2585 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471605    2585 flags.go:64] FLAG: --max-pods="110"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471608    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471611    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471614    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 11:14:25.478872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471617    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471620    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471623    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471626    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471633    2585 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471636    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471639    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471643    2585 flags.go:64] FLAG: --pod-cidr=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471646    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471652    2585 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471655    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471658    2585 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471661    2585 flags.go:64] FLAG: --port="10250"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471664    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471667    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f3224b2d1846b8e0"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471670    2585 flags.go:64] FLAG: --qos-reserved=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471674    2585 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471677    2585 flags.go:64] FLAG: --register-node="true"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471680    2585 flags.go:64] FLAG: --register-with-taints=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471683    2585 flags.go:64] FLAG: --registry-burst="10"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471686    2585 flags.go:64] FLAG: --registry-qps="5"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471689    2585 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471692    2585 flags.go:64] FLAG: --reserved-memory=""
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471695    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471698    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 11:14:25.479960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471701    2585 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471704    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471707    2585 flags.go:64] FLAG: --runonce="false"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471710    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471713    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471716    2585 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471719    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471722    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471725    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471728    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471731    2585 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471734    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471736    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471739    2585 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471743    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471746    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471750    2585 flags.go:64] FLAG: --system-cgroups=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471753    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471758    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471761    2585 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471763    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471767    2585 flags.go:64] FLAG: --tls-min-version=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471770    2585 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471774    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471777    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 11:14:25.481064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471779    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471782    2585 flags.go:64] FLAG: --v="2"
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471786    2585 flags.go:64] FLAG: --version="false"
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471790    2585 flags.go:64] FLAG: --vmodule=""
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471793    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.471796    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471908    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471912    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471915    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471917    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471921    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471924    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471927    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471930    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471932    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471935    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471938    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471940    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471943    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471946    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471949    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.482026 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471953    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471956    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471958    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471962    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471967    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471969    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471972    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471975    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471977    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471981    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471984    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471987    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471989    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471992    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471994    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.471997    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472000    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472003    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472007    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.482838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472010    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472015    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472019    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472022    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472025    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472028    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472030    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472033    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472036    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472038    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472041    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472044    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472046    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472050    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472053    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472055    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472058    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472061    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472063    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.483681 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472066    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472069    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472071    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472075    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472078    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472080    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472083    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472085    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472088    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472091    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472093    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472096    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472098    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472101    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472103    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472106    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472109    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472111    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472114    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472116    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.484209 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472119    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472122    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472124    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472127    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472129    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472132    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472135    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472138    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472140    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472143    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472145    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472148    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472151    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472153    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472156    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472160    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472163    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472165    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472168    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472171    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.484857 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472173    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472176    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472180    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472183    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472186    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.472189    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.472197    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.480217    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.480235    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.480262    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.480276    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480376    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480384    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480390    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.485383 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480395    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480402    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480408    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480413    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480418    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480422    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480427    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480431    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480435    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480441    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480446    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480450    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480454    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480458    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480462    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480466    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480471    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480475    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480479    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.485782 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480483    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480488    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480492    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480496    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480502    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480506    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480511    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480516    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480520    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480524    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480529    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480533    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480538    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480542    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480546    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480550    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480554    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480558    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480563    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.486252 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480567    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480571    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480575    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480580    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480585    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480590    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480593    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480597    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480602    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480606    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480610    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480614    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480618    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480623    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480627    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480631    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480637    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480642    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480647    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480653    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.486766 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480660    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480667    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480672    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480676    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480680    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480685    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480689    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480693    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480697    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480701    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480706    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480710    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480714    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480719    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480723    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480727    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480731    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480736    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480740    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480744    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.487271 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480748    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480753    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480757    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480761    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480765    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480769    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480773    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480776    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480780    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480785    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480789    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480793    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480798    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480802    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480806    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480810    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480814    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.487838 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.480818    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.480826    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481018    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481027    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481033    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481037    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481042    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481046    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481051    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481055    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481059    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481063    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481067    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481072    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481076    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 11:14:25.488269 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481080    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481083    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481087    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481090    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481094    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481099    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481102    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481107    2585 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481111    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481115    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481120    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481124    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481129    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481133    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481137    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481141    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481145    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481150    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481154    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481158    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 11:14:25.488736 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481162    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481167    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481171    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481175    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481179    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481183    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481187    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481191    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481195    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481200    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481204    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481216    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481223    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481228    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481232    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481237    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481241    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481246    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481250    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 11:14:25.489419 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481254    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481258    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481263    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481267    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481273    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481279    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481284    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481289    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481293    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481315    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481321    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481325    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481328    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481332    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481336    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481340    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481344    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481349    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481353    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 11:14:25.490217 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481357    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481361    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481365    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481369    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481373    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481378    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481383    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481387    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481391    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481396    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481400    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481404    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481409    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481413    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481416    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481420    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481425    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481429    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481433    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481440    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 11:14:25.491043 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481445    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481449    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481453    2585 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481459    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481463    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481467    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481471    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:25.481475    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.481483    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.481627    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.484506    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.485392    2585 server.go:1019] "Starting client certificate rotation"
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.485494    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.485534    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 11:14:25.492222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.489720    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 11:14:25.496797 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.496776    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 11:14:25.506237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.506220    2585 log.go:25] "Validated CRI v1 runtime API"
Sep 10 11:14:25.510237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.510220    2585 log.go:25] "Validated CRI v1 image API"
Sep 10 11:14:25.511119 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.511099    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 11:14:25.514886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.514869    2585 fs.go:135] Filesystem UUIDs: map[75aca4e0-272e-4a51-b4cd-c9fa1bb0cdb6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8d31100a-22b2-4968-a024-5cc059654b00:/dev/nvme0n1p4]
Sep 10 11:14:25.514937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.514887    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 11:14:25.519709 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.519605    2585 manager.go:216] Machine: {Timestamp:2026-09-10 11:14:25.51842555 +0000 UTC m=+0.322492888 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102352 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289f76d3c93d8c63bbf9377b3060d2 SystemUUID:ec289f76-d3c9-3d8c-63bb-f9377b3060d2 BootID:9c0651ac-7269-41eb-804b-d9d0266be1d2 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ef:62:16:2a:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ef:62:16:2a:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:3e:07:94:4f:d2 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 11:14:25.519709 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.519708    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 11:14:25.519807 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.519781    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 11:14:25.524214 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.524192    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 11:14:25.524359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.524217    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 11:14:25.525429 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.525418    2585 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 11:14:25.525462 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.525431    2585 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 11:14:25.525462 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.525439    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 11:14:25.526431 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.526421    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 11:14:25.527879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.527868    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 11:14:25.529820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.529805    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 11:14:25.530703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530691    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 11:14:25.530811 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530802    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 11:14:25.530844 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530831    2585 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 11:14:25.530844 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530839    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 11:14:25.530904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530853    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 11:14:25.530904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530862    2585 kubelet.go:398] "Adding apiserver pod source"
Sep 10 11:14:25.530904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.530870    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 11:14:25.531204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.531192    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 11:14:25.531239 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.531210    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 11:14:25.533171 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.533152    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 11:14:25.535795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.535782    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 11:14:25.535847 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.535812    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 11:14:25.536540 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536527    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536545    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536551    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536557    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536566    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536571    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536585    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536589    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 11:14:25.536600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536595    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 11:14:25.536831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536607    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 11:14:25.536831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536620    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 11:14:25.536831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.536625    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 11:14:25.537924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.537914    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 11:14:25.537924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.537925    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 11:14:25.541621 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.541607    2585 server.go:1268] "Started kubelet"
Sep 10 11:14:25.541706 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.541681    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 11:14:25.545006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.541770    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 11:14:25.545135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.545118    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 11:14:25.546334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.546114    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-86.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 11:14:25.546334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.546248    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 11:14:25.546179 ip-10-0-132-86 systemd[1]: Started Kubernetes Kubelet.
Sep 10 11:14:25.546976 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.546944    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 11:14:25.547125 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.547098    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 11:14:25.548357 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.548337    2585 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 11:14:25.549601 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.549578    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 11:14:25.552886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.552868    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 11:14:25.552975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.552887    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 11:14:25.553049 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.552991    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:25.553727 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.552207    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e95f94efd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.541582589 +0000 UTC m=+0.345649928,LastTimestamp:2026-09-10 11:14:25.541582589 +0000 UTC m=+0.345649928,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.553876 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.553857    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 11:14:25.553876 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.553865    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 11:14:25.553999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.553893    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 11:14:25.553999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.553941    2585 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 11:14:25.553999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.553951    2585 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 11:14:25.554414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554398    2585 factory.go:153] Registering CRI-O factory
Sep 10 11:14:25.554491 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554417    2585 factory.go:223] Registration of the crio container factory successfully
Sep 10 11:14:25.554491 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554468    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 11:14:25.554491 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554477    2585 factory.go:55] Registering systemd factory
Sep 10 11:14:25.554491 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554485    2585 factory.go:223] Registration of the systemd container factory successfully
Sep 10 11:14:25.554617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554500    2585 factory.go:103] Registering Raw factory
Sep 10 11:14:25.554617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554515    2585 manager.go:1196] Started watching for new ooms in manager
Sep 10 11:14:25.554899 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.554884    2585 manager.go:349] Starting recovery of all containers
Sep 10 11:14:25.564639 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.561968    2585 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 11:14:25.564639 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.562075    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 11:14:25.564864 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.564639    2585 manager.go:354] Recovery completed
Sep 10 11:14:25.570259 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.570244    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.573952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.573929    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.574026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.573963    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.574026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.573977    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.574960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.574944    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 11:14:25.574960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.574958    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 11:14:25.575077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.574974    2585 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 11:14:25.575077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.574979    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.576145 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.576130    2585 policy_none.go:49] "None policy: Start"
Sep 10 11:14:25.576181 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.576154    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 11:14:25.576181 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.576173    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 11:14:25.576895 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.576815    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e97e72b78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.57394828 +0000 UTC m=+0.378015620,LastTimestamp:2026-09-10 11:14:25.57394828 +0000 UTC m=+0.378015620,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.577191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.577177    2585 policy_none.go:47] "Start"
Sep 10 11:14:25.577991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.577955    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.578098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.578006    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.578098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.578022    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.590668 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.590606    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e97e77ea3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.573969571 +0000 UTC m=+0.378036912,LastTimestamp:2026-09-10 11:14:25.573969571 +0000 UTC m=+0.378036912,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.606021 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606007    2585 manager.go:337] "Starting Device Plugin manager"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.606041    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606054    2585 server.go:87] "Starting device plugin registration server"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606363    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606376    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.606436    2585 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e97e7b19c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-86.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.57398262 +0000 UTC m=+0.378049964,LastTimestamp:2026-09-10 11:14:25.57398262 +0000 UTC m=+0.378049964,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606496    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606667    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.606676    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.607868    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 11:14:25.620053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.607916    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:25.620510 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.620237    2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-86.ec2.internal.18d3f15e97e72b78\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e97e72b78  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.57394828 +0000 UTC m=+0.378015620,LastTimestamp:2026-09-10 11:14:25.577992728 +0000 UTC m=+0.382060077,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.621900 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.621884    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hjzvt"
Sep 10 11:14:25.631554 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.631538    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hjzvt"
Sep 10 11:14:25.631687 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.631631    2585 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-86.ec2.internal.18d3f15e97e77ea3\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-86.ec2.internal.18d3f15e97e77ea3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-86.ec2.internal,UID:ip-10-0-132-86.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-86.ec2.internal,},FirstTimestamp:2026-09-10 11:14:25.573969571 +0000 UTC m=+0.378036912,LastTimestamp:2026-09-10 11:14:25.578013012 +0000 UTC m=+0.382080352,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-86.ec2.internal,}"
Sep 10 11:14:25.655387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.655363    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 11:14:25.656459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.656440    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 11:14:25.656552 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.656489    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 11:14:25.656552 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.656509    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 11:14:25.656654 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.656588    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 11:14:25.659395 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.659377    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:14:25.707013 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.706993    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.707881 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.707866    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.707964 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.707896    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.707964 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.707910    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.707964 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.707939    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.719731 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.719650    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.719731 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.719674    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-86.ec2.internal\": node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:25.756991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.756971    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"]
Sep 10 11:14:25.757064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.757029    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.757920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.757903    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.757988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.757929    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.757988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.757939    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.759367 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.759355    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.759484 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.759470    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.759539 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.759506    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.760093 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760076    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.760161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760100    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.760161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760111    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.760161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760130    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.760161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760151    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.760277 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.760163    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.761352 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.761338    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.761414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.761362    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 11:14:25.762692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.762675    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 11:14:25.762767 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.762701    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 11:14:25.762767 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.762716    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeHasSufficientPID"
Sep 10 11:14:25.767011 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.766994    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:25.791846 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.791823    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-86.ec2.internal\" not found" node="ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.796013 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.795996    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-86.ec2.internal\" not found" node="ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.855793 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.855774    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9850f6ddb2c79a0bcf915bd203335015-config\") pod \"kube-apiserver-proxy-ip-10-0-132-86.ec2.internal\" (UID: \"9850f6ddb2c79a0bcf915bd203335015\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.855853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.855799    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.855853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.855817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.867877 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.867859    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:25.956252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956232    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.956359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956257    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9850f6ddb2c79a0bcf915bd203335015-config\") pod \"kube-apiserver-proxy-ip-10-0-132-86.ec2.internal\" (UID: \"9850f6ddb2c79a0bcf915bd203335015\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.956359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956274    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.956359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956326    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.956359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956353    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9850f6ddb2c79a0bcf915bd203335015-config\") pod \"kube-apiserver-proxy-ip-10-0-132-86.ec2.internal\" (UID: \"9850f6ddb2c79a0bcf915bd203335015\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.956510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:25.956356    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed5a4acc9d938a3e9ecfaf6efe466cf4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal\" (UID: \"ed5a4acc9d938a3e9ecfaf6efe466cf4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:25.968307 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:25.968289    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.069131 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.069070    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.095248 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.095228    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:26.099237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.099134    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:26.170137 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.170110    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.270713 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.270675    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.371278 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.371195    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.471457 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.471426    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-86.ec2.internal\" not found"
Sep 10 11:14:26.485617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.485596    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 11:14:26.485776 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.485759    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 11:14:26.503343 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.503301    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:14:26.532825 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.532793    2585 apiserver.go:52] "Watching apiserver"
Sep 10 11:14:26.544525 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.544497    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 11:14:26.547117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.546631    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-rhpz2","kube-system/global-pull-secret-syncer-bzrjs","kube-system/konnectivity-agent-xxcfc","openshift-dns/node-resolver-qv7xl","openshift-image-registry/node-ca-tf7ph","openshift-multus/multus-xzj2v","openshift-multus/network-metrics-daemon-wsmgl","openshift-network-diagnostics/network-check-target-jktjz","openshift-ovn-kubernetes/ovnkube-node-c4glv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r","openshift-cluster-node-tuning-operator/tuned-7t7xc","openshift-multus/multus-additional-cni-plugins-wjhd7"]
Sep 10 11:14:26.548722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.548693    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.549908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.549886    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.549999 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.549952    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:26.550958 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.550943    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.551615 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.551593    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.551700 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.551593    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.551700 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.551646    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-47rvq\""
Sep 10 11:14:26.551905 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.551889    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 11:14:26.552046 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.552032    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.552656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.552639    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 11:14:26.552946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.552931    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 11:14:26.553009 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.552972    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4v54n\""
Sep 10 11:14:26.553009 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.553000    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 11:14:26.553111 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.553060    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:26.553275 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.553251    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.554331 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.554297    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.555396 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555368    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.555504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555361    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7wt5d\""
Sep 10 11:14:26.555504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555416    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.555504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555455    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.555679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555508    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.555679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.555576    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.555679 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.555581    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:26.556631 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.556617    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:26.556689 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.556674    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:26.557216 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557199    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 11:14:26.557285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557192    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s2n6s\""
Sep 10 11:14:26.557285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557202    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 11:14:26.557285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557219    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.557456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557334    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 11:14:26.557588 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557572    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.557669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.557644    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6lxq2\""
Sep 10 11:14:26.558949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.558931    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-hostroot\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.558955    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-conf-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.558972    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvpt7\" (UniqueName: \"kubernetes.io/projected/9b294675-5652-4e12-b43b-828b7654f969-kube-api-access-cvpt7\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.558990    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-kubelet-config\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.559144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559009    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c14e1463-ba1c-4732-ad85-75536443232f-konnectivity-ca\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.559144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559033    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-hosts-file\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.559144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559082    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8967943b-d3b6-444f-a079-a06ed65e35c9-serviceca\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.559144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559124    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-socket-dir-parent\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559170    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-k8s-cni-cncf-io\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559187    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559203    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppbbz\" (UniqueName: \"kubernetes.io/projected/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-kube-api-access-ppbbz\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559234    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57b2\" (UniqueName: \"kubernetes.io/projected/8967943b-d3b6-444f-a079-a06ed65e35c9-kube-api-access-m57b2\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659c04c9-a0cf-4b9c-92be-2261a3aa2989-iptables-alerter-script\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.559349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559286    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c14e1463-ba1c-4732-ad85-75536443232f-agent-certs\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559378    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-netns\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559417    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-dbus\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559454    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-system-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559483    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-kubelet\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559520    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-multus-daemon-config\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559590    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-tmp-dir\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559641    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-cnibin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559667    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-multus\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559690    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-etc-kubernetes\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559706    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659c04c9-a0cf-4b9c-92be-2261a3aa2989-host-slash\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559727    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8967943b-d3b6-444f-a079-a06ed65e35c9-host\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559757    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-cni-binary-copy\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559786    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-bin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559872    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-multus-certs\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.559985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.559899    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8ppp\" (UniqueName: \"kubernetes.io/projected/659c04c9-a0cf-4b9c-92be-2261a3aa2989-kube-api-access-k8ppp\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.560499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.560480    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.561323 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.561302    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 11:14:26.561722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.561709    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.562450 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.562435    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 11:14:26.562916 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.562898    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.563640 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.563589    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.563745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.563727    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 11:14:26.563843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.563823    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 11:14:26.563843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564161    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.564456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564386    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.564456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564421    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.564456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564428    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 11:14:26.564660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564642    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vns8z\""
Sep 10 11:14:26.564928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.564905    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 11:14:26.565659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.565638    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 11:14:26.565738 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.565700    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-v28gg\""
Sep 10 11:14:26.565824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.565806    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-l7llh\""
Sep 10 11:14:26.566108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.566094    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dd46k\""
Sep 10 11:14:26.567037 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.567022    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 11:14:26.567111 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.567086    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 11:14:26.573417 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.573397    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 11:14:26.574560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.574539    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 11:14:26.574639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.574602    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal"]
Sep 10 11:14:26.574639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.574621    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"
Sep 10 11:14:26.602471 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.602455    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 11:14:26.602847 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.602827    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal"]
Sep 10 11:14:26.618123 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.618104    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6qdg"
Sep 10 11:14:26.629617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.629596    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g6qdg"
Sep 10 11:14:26.633521 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.633481    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 11:09:26 +0000 UTC" deadline="2028-05-03 04:28:30.384007723 +0000 UTC"
Sep 10 11:14:26.633521 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.633518    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14417h14m3.750492497s"
Sep 10 11:14:26.654853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.654831    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 11:14:26.660442 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660418    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-node-log\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.660525 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660457    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-bin\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.660525 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660491    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.660607 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660521    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-registration-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.660607 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660551    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2tt\" (UniqueName: \"kubernetes.io/projected/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kube-api-access-nt2tt\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.660688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660597    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.660688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660637    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-env-overrides\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.660688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.660688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660677    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.660838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660694    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.660838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660709    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-tmp\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.660838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660724    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.660838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-netns\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.660838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660800    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-netns\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660844    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-kubelet\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660887    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-config\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-log-socket\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-system-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660974    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-kubelet\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.660999    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661025    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-tmp-dir\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.661078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661068    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-kubelet\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-etc-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661083    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-system-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.661131    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661161    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovn-node-metrics-cert\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661189    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-socket-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.661211    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:27.161195754 +0000 UTC m=+1.965263087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661264    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661294    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-multus\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661334    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-etc-kubernetes\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661359    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-cni-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661362    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-multus\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-systemd-units\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.661438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661403    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-etc-kubernetes\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-ovn\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661478    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661508    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8ppp\" (UniqueName: \"kubernetes.io/projected/659c04c9-a0cf-4b9c-92be-2261a3aa2989-kube-api-access-k8ppp\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661534    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysconfig\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661561    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-hostroot\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661586    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvpt7\" (UniqueName: \"kubernetes.io/projected/9b294675-5652-4e12-b43b-828b7654f969-kube-api-access-cvpt7\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661608    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-hostroot\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661612    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-kubelet-config\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661668    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-kubelet-config\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661675    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-script-lib\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661705    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-var-lib-kubelet\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661729    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661760    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-socket-dir-parent\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661798    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-k8s-cni-cncf-io\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661843    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-socket-dir-parent\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppbbz\" (UniqueName: \"kubernetes.io/projected/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-kube-api-access-ppbbz\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.662065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661887    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-k8s-cni-cncf-io\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661916    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnw9r\" (UniqueName: \"kubernetes.io/projected/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-kube-api-access-nnw9r\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661947    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-sys-fs\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.661979    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-systemd\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662015    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-tmp-dir\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662018    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-host\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662042    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpnf\" (UniqueName: \"kubernetes.io/projected/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-kube-api-access-kmpnf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662077    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57b2\" (UniqueName: \"kubernetes.io/projected/8967943b-d3b6-444f-a079-a06ed65e35c9-kube-api-access-m57b2\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662107    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659c04c9-a0cf-4b9c-92be-2261a3aa2989-iptables-alerter-script\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662130    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-slash\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662155    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-var-lib-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662182    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgrgq\" (UniqueName: \"kubernetes.io/projected/755bead5-09ff-4756-a084-0f327c88a9e1-kube-api-access-wgrgq\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662307    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c14e1463-ba1c-4732-ad85-75536443232f-agent-certs\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-lib-modules\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662436    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662467    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-dbus\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662514    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60beda01-3d15-4bc8-9682-d5d86b363440-dbus\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:26.662879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662536    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-systemd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662555    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-kubernetes\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662570    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-tuned\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662593    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-multus-daemon-config\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-modprobe-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662622    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-conf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662660    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-cnibin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662686    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659c04c9-a0cf-4b9c-92be-2261a3aa2989-host-slash\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662711    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662727    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-cnibin\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662767    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8967943b-d3b6-444f-a079-a06ed65e35c9-host\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662790    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-cni-binary-copy\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662861    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662876    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-bin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-multus-certs\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662933    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662955    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-netd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.663795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.662983    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-sys\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663004    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-system-cni-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663036    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-conf-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663053    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c14e1463-ba1c-4732-ad85-75536443232f-konnectivity-ca\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663069    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-hosts-file\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663085    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-device-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663101    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-run\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663125    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m58z\" (UniqueName: \"kubernetes.io/projected/7cf2c07f-5632-49fc-85df-587b69a18319-kube-api-access-5m58z\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663143    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8967943b-d3b6-444f-a079-a06ed65e35c9-serviceca\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663165    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-netns\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663269    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-run-multus-certs\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663294    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8967943b-d3b6-444f-a079-a06ed65e35c9-host\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663302    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-cni-binary-copy\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663316    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-host-var-lib-cni-bin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663342    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-multus-conf-dir\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663364    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9b294675-5652-4e12-b43b-828b7654f969-multus-daemon-config\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663407    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659c04c9-a0cf-4b9c-92be-2261a3aa2989-host-slash\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663452    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b294675-5652-4e12-b43b-828b7654f969-cnibin\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.664679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663451    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-hosts-file\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.665676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663712    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659c04c9-a0cf-4b9c-92be-2261a3aa2989-iptables-alerter-script\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.665676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663785    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8967943b-d3b6-444f-a079-a06ed65e35c9-serviceca\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.665676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.663902    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c14e1463-ba1c-4732-ad85-75536443232f-konnectivity-ca\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.666614 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.666595    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c14e1463-ba1c-4732-ad85-75536443232f-agent-certs\") pod \"konnectivity-agent-xxcfc\" (UID: \"c14e1463-ba1c-4732-ad85-75536443232f\") " pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.670991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.670971    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8ppp\" (UniqueName: \"kubernetes.io/projected/659c04c9-a0cf-4b9c-92be-2261a3aa2989-kube-api-access-k8ppp\") pod \"iptables-alerter-rhpz2\" (UID: \"659c04c9-a0cf-4b9c-92be-2261a3aa2989\") " pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.672819 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.672793    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvpt7\" (UniqueName: \"kubernetes.io/projected/9b294675-5652-4e12-b43b-828b7654f969-kube-api-access-cvpt7\") pod \"multus-xzj2v\" (UID: \"9b294675-5652-4e12-b43b-828b7654f969\") " pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.673199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.673180    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppbbz\" (UniqueName: \"kubernetes.io/projected/ded2fd1b-1a87-4842-a072-82ee5c3fc4a2-kube-api-access-ppbbz\") pod \"node-resolver-qv7xl\" (UID: \"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2\") " pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.678148 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.678081    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57b2\" (UniqueName: \"kubernetes.io/projected/8967943b-d3b6-444f-a079-a06ed65e35c9-kube-api-access-m57b2\") pod \"node-ca-tf7ph\" (UID: \"8967943b-d3b6-444f-a079-a06ed65e35c9\") " pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.763461 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-lib-modules\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763474    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763505    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-systemd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763532    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-lib-modules\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763549    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-kubernetes\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763589    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-tuned\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763615    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-modprobe-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763621    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-kubernetes\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.763663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-conf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763668    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763674    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-systemd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-cnibin\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763697    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763725    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-modprobe-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763728    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763754    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-netd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763765    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763777    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-sys\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763802    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-system-cni-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763771    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-cnibin\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763817    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-netd\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763832    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-device-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763802    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-conf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763837    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-sys\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763865    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7cf2c07f-5632-49fc-85df-587b69a18319-system-cni-dir\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.764220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763868    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-run\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763889    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-device-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763908    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-run\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763908    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m58z\" (UniqueName: \"kubernetes.io/projected/7cf2c07f-5632-49fc-85df-587b69a18319-kube-api-access-5m58z\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-netns\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763985    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-run-netns\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.763973    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-node-log\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764044    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-node-log\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-bin\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764091    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764114    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-registration-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764161    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764168    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-registration-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764194    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2tt\" (UniqueName: \"kubernetes.io/projected/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kube-api-access-nt2tt\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764212    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-cni-bin\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764221    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764246    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-env-overrides\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.765210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764278    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764425    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764454    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764483    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-tmp\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764509    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764534    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-kubelet\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764559    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-config\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764572    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764579    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysctl-d\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764585    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-log-socket\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764626    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-log-socket\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764663    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-etc-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764690    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764714    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovn-node-metrics-cert\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764748    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-socket-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764765    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-env-overrides\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764776    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-systemd-units\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.766191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-ovn\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764862    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-systemd-units\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764896    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-etc-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764931    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-kubelet\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.764973    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-binary-copy\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765041    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765050    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-socket-dir\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765083    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765114    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysconfig\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765171    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-run-ovn\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765215    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-script-lib\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765239    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7cf2c07f-5632-49fc-85df-587b69a18319-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765247    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765258    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-var-lib-kubelet\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765289    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-sysconfig\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765306    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-var-lib-kubelet\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765311    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765346    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnw9r\" (UniqueName: \"kubernetes.io/projected/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-kube-api-access-nnw9r\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765371    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-sys-fs\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765414    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-config\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765420    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-systemd\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.765430    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-host\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765469    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpnf\" (UniqueName: \"kubernetes.io/projected/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-kube-api-access-kmpnf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765479    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-sys-fs\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.765495    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:27.265476517 +0000 UTC m=+2.069543861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765527    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-slash\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-var-lib-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgrgq\" (UniqueName: \"kubernetes.io/projected/755bead5-09ff-4756-a084-0f327c88a9e1-kube-api-access-wgrgq\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765669    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-systemd\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765716    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-host-slash\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765748    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-var-lib-openvswitch\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765769    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-host\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.767657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.765932    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovnkube-script-lib\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.768199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.766283    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-etc-tuned\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.768199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.767071    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-tmp\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.768199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.768103    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-ovn-node-metrics-cert\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.770673 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.770656    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:26.770759 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.770678    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:26.770759 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.770687    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:26.770759 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:26.770744    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:27.270729176 +0000 UTC m=+2.074796533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:26.773320 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.773292    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m58z\" (UniqueName: \"kubernetes.io/projected/7cf2c07f-5632-49fc-85df-587b69a18319-kube-api-access-5m58z\") pod \"multus-additional-cni-plugins-wjhd7\" (UID: \"7cf2c07f-5632-49fc-85df-587b69a18319\") " pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:26.774918 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.774895    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnw9r\" (UniqueName: \"kubernetes.io/projected/0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b-kube-api-access-nnw9r\") pod \"ovnkube-node-c4glv\" (UID: \"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b\") " pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.775703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.775649    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpnf\" (UniqueName: \"kubernetes.io/projected/f88ee55e-2b54-42f9-8fbd-4783a5589cd6-kube-api-access-kmpnf\") pod \"tuned-7t7xc\" (UID: \"f88ee55e-2b54-42f9-8fbd-4783a5589cd6\") " pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.775822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.775750    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2tt\" (UniqueName: \"kubernetes.io/projected/9aa8763b-d07b-4cbf-bd45-ff3eed911ea2-kube-api-access-nt2tt\") pod \"aws-ebs-csi-driver-node-2799r\" (UID: \"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.778068 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.778048    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgrgq\" (UniqueName: \"kubernetes.io/projected/755bead5-09ff-4756-a084-0f327c88a9e1-kube-api-access-wgrgq\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:26.875198 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.875164    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5a4acc9d938a3e9ecfaf6efe466cf4.slice/crio-39351d6f2b035f864f77bfeed8b261f314ccc3432eb64f170beb8da95a4cade0 WatchSource:0}: Error finding container 39351d6f2b035f864f77bfeed8b261f314ccc3432eb64f170beb8da95a4cade0: Status 404 returned error can't find the container with id 39351d6f2b035f864f77bfeed8b261f314ccc3432eb64f170beb8da95a4cade0
Sep 10 11:14:26.878355 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.878331    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rhpz2"
Sep 10 11:14:26.880282 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.880265    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:14:26.886709 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.886689    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659c04c9_a0cf_4b9c_92be_2261a3aa2989.slice/crio-59701ab7e0053a23530d5d68fece7fca0a216dcbb8730ee8603412ba6b904ee5 WatchSource:0}: Error finding container 59701ab7e0053a23530d5d68fece7fca0a216dcbb8730ee8603412ba6b904ee5: Status 404 returned error can't find the container with id 59701ab7e0053a23530d5d68fece7fca0a216dcbb8730ee8603412ba6b904ee5
Sep 10 11:14:26.896177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.896157    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:26.904015 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.903995    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14e1463_ba1c_4732_ad85_75536443232f.slice/crio-a2ab74efeaea10a754a56bf1a6f215709455477598c7c23eba76827b1a6e0cb3 WatchSource:0}: Error finding container a2ab74efeaea10a754a56bf1a6f215709455477598c7c23eba76827b1a6e0cb3: Status 404 returned error can't find the container with id a2ab74efeaea10a754a56bf1a6f215709455477598c7c23eba76827b1a6e0cb3
Sep 10 11:14:26.914442 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.914425    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qv7xl"
Sep 10 11:14:26.915150 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.915133    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:14:26.921576 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.921551    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded2fd1b_1a87_4842_a072_82ee5c3fc4a2.slice/crio-71d8972faca1497e969b28b1d187f63a4d1d49fb6df3b57218dd51864bf5e8f0 WatchSource:0}: Error finding container 71d8972faca1497e969b28b1d187f63a4d1d49fb6df3b57218dd51864bf5e8f0: Status 404 returned error can't find the container with id 71d8972faca1497e969b28b1d187f63a4d1d49fb6df3b57218dd51864bf5e8f0
Sep 10 11:14:26.929774 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.929751    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tf7ph"
Sep 10 11:14:26.935504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.935488    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xzj2v"
Sep 10 11:14:26.946702 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.946682    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8967943b_d3b6_444f_a079_a06ed65e35c9.slice/crio-b9550a33d83772891a5ff4828304ac9992e72f085467088277e7b112fc391c0c WatchSource:0}: Error finding container b9550a33d83772891a5ff4828304ac9992e72f085467088277e7b112fc391c0c: Status 404 returned error can't find the container with id b9550a33d83772891a5ff4828304ac9992e72f085467088277e7b112fc391c0c
Sep 10 11:14:26.947315 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.947282    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b294675_5652_4e12_b43b_828b7654f969.slice/crio-96b42dbf0c7df0fb059fd454f9abee6dde50a3f0ddc832a3bbf6f2b8fdd7dabf WatchSource:0}: Error finding container 96b42dbf0c7df0fb059fd454f9abee6dde50a3f0ddc832a3bbf6f2b8fdd7dabf: Status 404 returned error can't find the container with id 96b42dbf0c7df0fb059fd454f9abee6dde50a3f0ddc832a3bbf6f2b8fdd7dabf
Sep 10 11:14:26.947999 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.947981    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9850f6ddb2c79a0bcf915bd203335015.slice/crio-0d33a7ff40cf4028067708e12164099e125482b27dda00c33898418f57c3b9a7 WatchSource:0}: Error finding container 0d33a7ff40cf4028067708e12164099e125482b27dda00c33898418f57c3b9a7: Status 404 returned error can't find the container with id 0d33a7ff40cf4028067708e12164099e125482b27dda00c33898418f57c3b9a7
Sep 10 11:14:26.959970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.959952    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:26.966187 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.966168    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r"
Sep 10 11:14:26.967584 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.967564    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcdf0eb_13b7_4f26_b45d_1c3270e3d28b.slice/crio-ed7c5b3b8b3ebfb9dcbaf1ab416d67a0ade09cb6a4539c919f54c77e243d392e WatchSource:0}: Error finding container ed7c5b3b8b3ebfb9dcbaf1ab416d67a0ade09cb6a4539c919f54c77e243d392e: Status 404 returned error can't find the container with id ed7c5b3b8b3ebfb9dcbaf1ab416d67a0ade09cb6a4539c919f54c77e243d392e
Sep 10 11:14:26.973544 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:26.973519    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa8763b_d07b_4cbf_bd45_ff3eed911ea2.slice/crio-31a8142c9c6c31866bdb78402b4aefafaba14f7d622bf51b248bb30bfe366694 WatchSource:0}: Error finding container 31a8142c9c6c31866bdb78402b4aefafaba14f7d622bf51b248bb30bfe366694: Status 404 returned error can't find the container with id 31a8142c9c6c31866bdb78402b4aefafaba14f7d622bf51b248bb30bfe366694
Sep 10 11:14:26.995394 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.995373    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7t7xc"
Sep 10 11:14:26.998950 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:26.998931    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wjhd7"
Sep 10 11:14:27.003136 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:27.003117    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88ee55e_2b54_42f9_8fbd_4783a5589cd6.slice/crio-bf274f207395e56a54fadcbab7258b82480cceb9caaf330eee1fb0f0ddcf0cee WatchSource:0}: Error finding container bf274f207395e56a54fadcbab7258b82480cceb9caaf330eee1fb0f0ddcf0cee: Status 404 returned error can't find the container with id bf274f207395e56a54fadcbab7258b82480cceb9caaf330eee1fb0f0ddcf0cee
Sep 10 11:14:27.007267 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:14:27.007245    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf2c07f_5632_49fc_85df_587b69a18319.slice/crio-141d7e7da62002b5f2602832e93fb8f87ca5377405055b6e5f96541c2536cf42 WatchSource:0}: Error finding container 141d7e7da62002b5f2602832e93fb8f87ca5377405055b6e5f96541c2536cf42: Status 404 returned error can't find the container with id 141d7e7da62002b5f2602832e93fb8f87ca5377405055b6e5f96541c2536cf42
Sep 10 11:14:27.008896 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.008877    2585 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-wjhd7_openshift-multus(7cf2c07f-5632-49fc-85df-587b69a18319): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 10 11:14:27.008947 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.008909    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" podUID="7cf2c07f-5632-49fc-85df-587b69a18319"
Sep 10 11:14:27.046074 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.046051    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:14:27.168999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.168972    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:27.169167 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.169077    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:27.169167 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.169132    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:28.169117823 +0000 UTC m=+2.973185151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:27.271803 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.269793    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:27.271803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.270049    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:27.271803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.270121    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:28.270095647 +0000 UTC m=+3.074162994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:27.371165 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.370896    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:27.371165 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.371077    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:27.371165 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.371108    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:27.371165 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.371123    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:27.371165 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.371191    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:28.371170175 +0000 UTC m=+3.175237504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:27.444018 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.443941    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 11:14:27.632577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.632501    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:09:27 +0000 UTC" deadline="2028-06-02 05:18:26.996404586 +0000 UTC"
Sep 10 11:14:27.632577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.632574    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15138h3m59.363835213s"
Sep 10 11:14:27.666406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.666280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xzj2v" event={"ID":"9b294675-5652-4e12-b43b-828b7654f969","Type":"ContainerStarted","Data":"96b42dbf0c7df0fb059fd454f9abee6dde50a3f0ddc832a3bbf6f2b8fdd7dabf"}
Sep 10 11:14:27.669255 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.669197    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tf7ph" event={"ID":"8967943b-d3b6-444f-a079-a06ed65e35c9","Type":"ContainerStarted","Data":"b9550a33d83772891a5ff4828304ac9992e72f085467088277e7b112fc391c0c"}
Sep 10 11:14:27.672694 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.672640    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xxcfc" event={"ID":"c14e1463-ba1c-4732-ad85-75536443232f","Type":"ContainerStarted","Data":"a2ab74efeaea10a754a56bf1a6f215709455477598c7c23eba76827b1a6e0cb3"}
Sep 10 11:14:27.680693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.680587    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhpz2" event={"ID":"659c04c9-a0cf-4b9c-92be-2261a3aa2989","Type":"ContainerStarted","Data":"59701ab7e0053a23530d5d68fece7fca0a216dcbb8730ee8603412ba6b904ee5"}
Sep 10 11:14:27.696916 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.696811    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerStarted","Data":"141d7e7da62002b5f2602832e93fb8f87ca5377405055b6e5f96541c2536cf42"}
Sep 10 11:14:27.703907 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:27.703666    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0d2fae617617ccbdef514e3068eb3db7ed6f5869fe748bbfc9d66eb536d3e26f\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" podUID="7cf2c07f-5632-49fc-85df-587b69a18319"
Sep 10 11:14:27.719083 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.718998    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7t7xc" event={"ID":"f88ee55e-2b54-42f9-8fbd-4783a5589cd6","Type":"ContainerStarted","Data":"bf274f207395e56a54fadcbab7258b82480cceb9caaf330eee1fb0f0ddcf0cee"}
Sep 10 11:14:27.740254 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.740158    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r" event={"ID":"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2","Type":"ContainerStarted","Data":"31a8142c9c6c31866bdb78402b4aefafaba14f7d622bf51b248bb30bfe366694"}
Sep 10 11:14:27.752162 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.752131    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"ed7c5b3b8b3ebfb9dcbaf1ab416d67a0ade09cb6a4539c919f54c77e243d392e"}
Sep 10 11:14:27.775740 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.775711    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qv7xl" event={"ID":"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2","Type":"ContainerStarted","Data":"71d8972faca1497e969b28b1d187f63a4d1d49fb6df3b57218dd51864bf5e8f0"}
Sep 10 11:14:27.808661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.806158    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal" event={"ID":"ed5a4acc9d938a3e9ecfaf6efe466cf4","Type":"ContainerStarted","Data":"39351d6f2b035f864f77bfeed8b261f314ccc3432eb64f170beb8da95a4cade0"}
Sep 10 11:14:27.814757 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:27.814727    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal" event={"ID":"9850f6ddb2c79a0bcf915bd203335015","Type":"ContainerStarted","Data":"0d33a7ff40cf4028067708e12164099e125482b27dda00c33898418f57c3b9a7"}
Sep 10 11:14:28.178472 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.178418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:28.178669 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.178645    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:28.178736 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.178710    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:30.178691417 +0000 UTC m=+4.982758772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:28.278898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.278849    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:28.279086 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.279066    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:28.279154 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.279130    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:30.27911009 +0000 UTC m=+5.083177423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:28.379803 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.379398    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:28.379803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.379574    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:28.379803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.379594    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:28.379803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.379607    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:28.379803 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.379667    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:30.379648395 +0000 UTC m=+5.183715739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:28.636584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.636494    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 11:09:27 +0000 UTC" deadline="2028-05-31 13:13:09.870509478 +0000 UTC"
Sep 10 11:14:28.636584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.636535    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15097h58m41.233979664s"
Sep 10 11:14:28.657774 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.657743    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:28.657937 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.657877    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:28.658320 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.658282    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:28.658433 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.658407    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:28.658501 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:28.658485    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:28.659010 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:28.658577    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:30.200975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.200941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:30.201475 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.201133    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:30.201475 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.201193    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:34.201178754 +0000 UTC m=+9.005246084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:30.302825 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.302188    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:30.302825 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.302406    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:30.302825 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.302478    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:34.302459385 +0000 UTC m=+9.106526717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:30.403385 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.403304    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:30.403594 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.403524    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:30.403594 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.403550    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:30.403594 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.403563    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:30.403764 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.403625    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:34.403605484 +0000 UTC m=+9.207672834 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:30.657317 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.657264    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:30.657317 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.657276    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:30.657533 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.657406    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:30.657587 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.657526    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:30.657587 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:30.657572    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:30.657665 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:30.657640    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:32.657159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:32.657122    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:32.657159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:32.657155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:32.657746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:32.657260    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:32.657746 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:32.657258    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:32.657746 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:32.657417    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:32.657746 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:32.657518    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:34.240031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.239959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:34.240530 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.240103    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:34.240530 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.240165    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:42.240147554 +0000 UTC m=+17.044214897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:34.341265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.341239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:34.341437 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.341397    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:34.341502 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.341453    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:42.341434691 +0000 UTC m=+17.145502018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:34.442025 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.441990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:34.442230 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.442155    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:34.442230 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.442174    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:34.442230 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.442185    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:34.442415 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.442244    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:42.442225573 +0000 UTC m=+17.246292918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:34.657610 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.657523    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:34.657777 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.657705    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:34.657777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.657718    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:34.657898 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.657801    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:34.657898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:34.657825    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:34.657988 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:34.657898    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:36.657370 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:36.657343    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:36.657835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:36.657343    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:36.657835 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:36.657462    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:36.657835 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:36.657562    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:36.657835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:36.657343    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:36.657835 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:36.657656    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:38.656970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:38.656929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:38.657419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:38.656977    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:38.657419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:38.656937    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:38.657419 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:38.657057    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:38.657419 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:38.657150    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:38.657419 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:38.657210    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:40.656859 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:40.656827    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:40.657246 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:40.656946    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:40.657246 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:40.656953    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:40.657246 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:40.656981    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:40.657246 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:40.657073    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:40.657246 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:40.657170    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:42.304204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.304164    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:42.304768 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.304335    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:42.304768 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.304402    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:58.304383457 +0000 UTC m=+33.108450786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:42.405047 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.405009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:42.405216 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.405164    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:42.405268 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.405237    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:58.405221557 +0000 UTC m=+33.209288883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:42.505601 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.505556    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:42.505763 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.505722    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:42.505763 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.505747    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:42.505763 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.505760    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:42.505931 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.505822    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:14:58.505804402 +0000 UTC m=+33.309871745 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:42.657184 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.657109    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:42.657370 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.657201    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:42.657370 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:42.657323    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:42.657370 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.657348    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:42.657541 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.657443    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:42.657541 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:42.657520    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:44.656953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:44.656915    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:44.656953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:44.656950    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:44.657413 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:44.656950    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:44.657413 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:44.657030    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:44.657413 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:44.657146    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:44.657413 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:44.657232    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:46.657923 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.657609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:46.658510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.657609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:46.658510 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:46.658015    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:46.658510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.657609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:46.658510 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:46.658079    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:46.658510 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:46.658174    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:46.862973 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.862890    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qv7xl" event={"ID":"ded2fd1b-1a87-4842-a072-82ee5c3fc4a2","Type":"ContainerStarted","Data":"1738151aea0b1d17b0aedb05ef354b016feb2a06ec12e7bbe876a139d793d2e4"}
Sep 10 11:14:46.864525 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.864498    2585 generic.go:356] "Generic (PLEG): container finished" podID="ed5a4acc9d938a3e9ecfaf6efe466cf4" containerID="8514e007810f0245eddaa7bd00c047b8bd8b8a2b860c15ac6153481638028e6d" exitCode=0
Sep 10 11:14:46.864645 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.864568    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal" event={"ID":"ed5a4acc9d938a3e9ecfaf6efe466cf4","Type":"ContainerDied","Data":"8514e007810f0245eddaa7bd00c047b8bd8b8a2b860c15ac6153481638028e6d"}
Sep 10 11:14:46.866001 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.865974    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal" event={"ID":"9850f6ddb2c79a0bcf915bd203335015","Type":"ContainerStarted","Data":"f6c86b365f891485a78d83311df25a06bfea3c4875800cf1801d3bc6c36cf14f"}
Sep 10 11:14:46.867557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.867538    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xzj2v" event={"ID":"9b294675-5652-4e12-b43b-828b7654f969","Type":"ContainerStarted","Data":"c7abaf396950833eed991249a3012f6aea80b5ace4376e8d316b8183e962229b"}
Sep 10 11:14:46.869057 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.869040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tf7ph" event={"ID":"8967943b-d3b6-444f-a079-a06ed65e35c9","Type":"ContainerStarted","Data":"5e349d6b90bbcb18bd3aba237d01a6495c4e0d04926ba4c0f951348c66fbd7e3"}
Sep 10 11:14:46.870493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.870471    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xxcfc" event={"ID":"c14e1463-ba1c-4732-ad85-75536443232f","Type":"ContainerStarted","Data":"140f13dc129d37101a7f948a422a742d38ea5c9ad7ea8390b03cd026b9100f1b"}
Sep 10 11:14:46.872024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.872005    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="8919dabd7683876359578b6a2581536bcddea25bb2d1d370468577c9cc26e71f" exitCode=0
Sep 10 11:14:46.872101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.872072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"8919dabd7683876359578b6a2581536bcddea25bb2d1d370468577c9cc26e71f"}
Sep 10 11:14:46.873549 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.873514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7t7xc" event={"ID":"f88ee55e-2b54-42f9-8fbd-4783a5589cd6","Type":"ContainerStarted","Data":"5dba838cf514e26dfd4ca21cda78c4567c53865facc25f635c2f72177829866c"}
Sep 10 11:14:46.875057 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.875038    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r" event={"ID":"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2","Type":"ContainerStarted","Data":"9b40caced1de74ad33ad1e53e22cd82a018d6c24b168f3a4766c0b3b6907e924"}
Sep 10 11:14:46.877858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.877841    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:14:46.878211 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878190    2585 generic.go:356] "Generic (PLEG): container finished" podID="0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b" containerID="1169667c3d4c1b54d5142a984854736721414be3ee3f494ff40a58de6de55e35" exitCode=1
Sep 10 11:14:46.878269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878218    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"5345783bd9ca180235f9270bfce83b6ba57e9b64dda4292874fc15f0f63e6166"}
Sep 10 11:14:46.878269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878266    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"c443d68f9f45cdc47f3b2152424e854fd1521d91fd9e52f35c6f7316ef59fdbc"}
Sep 10 11:14:46.878354 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878296    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"9247230f761f91bccf060b354be18bf1c6745f8f372f6addb956c58e6aecec3e"}
Sep 10 11:14:46.878354 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878344    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"ddd6b74c98bb5c59f8127a2d476f245e2b7e701fcd6ff96dfd93f42e905327bb"}
Sep 10 11:14:46.878414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878371    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerDied","Data":"1169667c3d4c1b54d5142a984854736721414be3ee3f494ff40a58de6de55e35"}
Sep 10 11:14:46.878414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.878399    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"c708ae6d82e638e06549cb2c0f68b6d6d1a314489897a40d9756ad38ed9cfab8"}
Sep 10 11:14:46.918905 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.918858    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qv7xl" podStartSLOduration=1.940357281 podStartE2EDuration="20.918841675s" totalImagesPullingTime="18.978484394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:46.892959215 +0000 UTC m=+21.697026563" watchObservedRunningTime="2026-09-10 11:14:46.918841675 +0000 UTC m=+21.722909023"
Sep 10 11:14:46.942568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.942510    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-86.ec2.internal" podStartSLOduration=19.942486648 podStartE2EDuration="19.942486648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:46.921410284 +0000 UTC m=+21.725477631" watchObservedRunningTime="2026-09-10 11:14:46.942486648 +0000 UTC m=+21.746553996"
Sep 10 11:14:46.969287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.969133    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xxcfc" podStartSLOduration=1.972725015 podStartE2EDuration="20.969113736s" totalImagesPullingTime="18.996388721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:46.942834129 +0000 UTC m=+21.746901477" watchObservedRunningTime="2026-09-10 11:14:46.969113736 +0000 UTC m=+21.773181062"
Sep 10 11:14:46.969441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:46.969346    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tf7ph" podStartSLOduration=2.015748907 podStartE2EDuration="20.969334133s" totalImagesPullingTime="18.953585226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:46.969088604 +0000 UTC m=+21.773155956" watchObservedRunningTime="2026-09-10 11:14:46.969334133 +0000 UTC m=+21.773401481"
Sep 10 11:14:47.113098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.113016    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xzj2v" podStartSLOduration=2.157835515 podStartE2EDuration="21.113000516s" totalImagesPullingTime="18.955165001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:47.042227021 +0000 UTC m=+21.846294370" watchObservedRunningTime="2026-09-10 11:14:47.113000516 +0000 UTC m=+21.917067895"
Sep 10 11:14:47.232956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.232882    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7t7xc" podStartSLOduration=2.33572668 podStartE2EDuration="21.232861493s" totalImagesPullingTime="18.897134813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:47.170036596 +0000 UTC m=+21.974103944" watchObservedRunningTime="2026-09-10 11:14:47.232861493 +0000 UTC m=+22.036928841"
Sep 10 11:14:47.784685 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.784646    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 11:14:47.882705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.882664    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal" event={"ID":"ed5a4acc9d938a3e9ecfaf6efe466cf4","Type":"ContainerStarted","Data":"266be16d526ad79f7d124410c2357820481e10947256748a15919adfd4f37e51"}
Sep 10 11:14:47.884463 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.884433    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rhpz2" event={"ID":"659c04c9-a0cf-4b9c-92be-2261a3aa2989","Type":"ContainerStarted","Data":"b054af30fe9b4b54c8218e366145bf3342487e81733579db9f01628e2199ea05"}
Sep 10 11:14:47.886708 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.886666    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r" event={"ID":"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2","Type":"ContainerStarted","Data":"9a92ba0d6e24e0a0dfa57fedd8c6a799210e5eed36da1e024a2ee32b187510d1"}
Sep 10 11:14:47.909596 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.909547    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-86.ec2.internal" podStartSLOduration=20.909532439 podStartE2EDuration="20.909532439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:47.909354694 +0000 UTC m=+22.713422043" watchObservedRunningTime="2026-09-10 11:14:47.909532439 +0000 UTC m=+22.713599790"
Sep 10 11:14:47.943349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:47.943270    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rhpz2" podStartSLOduration=2.929333535 podStartE2EDuration="21.943257285s" totalImagesPullingTime="19.01392375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:47.94197105 +0000 UTC m=+22.746038397" watchObservedRunningTime="2026-09-10 11:14:47.943257285 +0000 UTC m=+22.747324686"
Sep 10 11:14:48.619746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.619642    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T11:14:47.784667796Z","UUID":"e8d6a416-546f-43a6-9516-419a079d3e1a","Handler":null,"Name":"","Endpoint":""}
Sep 10 11:14:48.621153 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.621126    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 11:14:48.621332 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.621164    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 11:14:48.657464 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.657439    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:48.657572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.657540    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:48.657572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.657548    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:48.657685 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:48.657544    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:48.657685 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:48.657650    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:48.657788 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:48.657751    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:48.892523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.892455    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:14:48.892908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:48.892878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"ad0d5a7305347841d9ef97ff34b4ca0801ccb821d3384f7b0db7f1daa480a080"}
Sep 10 11:14:49.727500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.727235    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:49.727934 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.727913    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:49.898017 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.897938    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r" event={"ID":"9aa8763b-d07b-4cbf-bd45-ff3eed911ea2","Type":"ContainerStarted","Data":"cba429d0c7aaede0ca9581345c9ff595ece61d6144d861a2db48ee91ea11023e"}
Sep 10 11:14:49.898477 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.898109    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:49.898730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.898702    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xxcfc"
Sep 10 11:14:49.930959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:49.930905    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2799r" podStartSLOduration=1.869575802 podStartE2EDuration="23.930884744s" totalImagesPullingTime="22.061308942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:49.930784863 +0000 UTC m=+24.734852214" watchObservedRunningTime="2026-09-10 11:14:49.930884744 +0000 UTC m=+24.734952093"
Sep 10 11:14:50.657530 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:50.657497    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:50.657715 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:50.657537    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:50.657715 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:50.657606    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:50.657715 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:50.657622    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:50.657715 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:50.657702    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:50.657927 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:50.657798    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:52.657796 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.657625    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:52.658160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.657686    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:52.658160 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:52.657864    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:52.658160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.657686    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:52.658160 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:52.657938    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:52.658160 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:52.658004    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:52.906024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.905996    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="5fb809aa226018660c28daec8d3c6fcb656a187648b5cc750a4ea3b854cfa45c" exitCode=0
Sep 10 11:14:52.906196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.906079    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"5fb809aa226018660c28daec8d3c6fcb656a187648b5cc750a4ea3b854cfa45c"}
Sep 10 11:14:52.909899 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.909879    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:14:52.910337 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.910315    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"278db7fd68d1da9399e3fdb594a721fdc10d14bcf9c2f546c7508eed2d903def"}
Sep 10 11:14:52.910562 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.910548    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:52.910618 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.910599    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:52.910710 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.910693    2585 scope.go:117] "RemoveContainer" containerID="1169667c3d4c1b54d5142a984854736721414be3ee3f494ff40a58de6de55e35"
Sep 10 11:14:52.926425 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.926405    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:52.926520 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:52.926505    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:53.829989 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.829957    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzrjs"]
Sep 10 11:14:53.830725 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.830084    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:53.830725 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:53.830187    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:53.833246 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.833225    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wsmgl"]
Sep 10 11:14:53.833418 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.833403    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:53.833679 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:53.833528    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:53.835267 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.835245    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jktjz"]
Sep 10 11:14:53.835386 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.835350    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:53.835456 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:53.835437    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:53.915489 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.915467    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:14:53.915806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.915785    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" event={"ID":"0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b","Type":"ContainerStarted","Data":"f7bb5d7334b9a03626c7d78c6ee76892d9e910d9411c1abe19a5e7bec7822ba9"}
Sep 10 11:14:53.916024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.916010    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:14:53.950221 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:53.950180    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv" podStartSLOduration=9.016569699 podStartE2EDuration="27.95016604s" totalImagesPullingTime="18.933596341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:14:53.949048378 +0000 UTC m=+28.753115727" watchObservedRunningTime="2026-09-10 11:14:53.95016604 +0000 UTC m=+28.754233427"
Sep 10 11:14:54.920921 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:54.920891    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="5df9db704878f19f0eaaad02c742aa178e47429200de6f6aa820a30ab423cf40" exitCode=0
Sep 10 11:14:54.921373 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:54.920959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"5df9db704878f19f0eaaad02c742aa178e47429200de6f6aa820a30ab423cf40"}
Sep 10 11:14:55.658624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:55.658587    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:55.658797 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:55.658707    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:55.658998 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:55.658978    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:55.659113 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:55.659094    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:55.659177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:55.659140    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:55.659249 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:55.659230    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:56.927637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:56.927452    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="e9801d0b196f5596b4436991e2ef6aa94bc44a685a5d27451a9f5ffe713453e2" exitCode=0
Sep 10 11:14:56.928082 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:56.927528    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"e9801d0b196f5596b4436991e2ef6aa94bc44a685a5d27451a9f5ffe713453e2"}
Sep 10 11:14:57.663607 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:57.663578    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:57.663778 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:57.663617    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:57.663778 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:57.663617    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:57.663778 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:57.663726    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jktjz" podUID="58c9df95-f1b3-491b-9152-3b14cdf94c93"
Sep 10 11:14:57.663911 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:57.663802    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzrjs" podUID="60beda01-3d15-4bc8-9682-d5d86b363440"
Sep 10 11:14:57.663911 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:57.663873    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wsmgl" podUID="755bead5-09ff-4756-a084-0f327c88a9e1"
Sep 10 11:14:58.319846 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:58.319812    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:58.320235 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.319943    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:58.320235 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.320006    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret podName:60beda01-3d15-4bc8-9682-d5d86b363440 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:30.319984958 +0000 UTC m=+65.124052293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret") pod "global-pull-secret-syncer-bzrjs" (UID: "60beda01-3d15-4bc8-9682-d5d86b363440") : object "kube-system"/"original-pull-secret" not registered
Sep 10 11:14:58.420888 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:58.420852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:58.421053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.420963    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:58.421053 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.421021    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:30.421006468 +0000 UTC m=+65.225073798 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 11:14:58.521242 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:58.521211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:58.521423 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.521364    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 11:14:58.521423 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.521379    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 11:14:58.521423 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.521388    2585 projected.go:196] Error preparing data for projected volume kube-api-access-59fbf for pod openshift-network-diagnostics/network-check-target-jktjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:58.521546 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:58.521443    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf podName:58c9df95-f1b3-491b-9152-3b14cdf94c93 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:30.521428493 +0000 UTC m=+65.325495819 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-59fbf" (UniqueName: "kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf") pod "network-check-target-jktjz" (UID: "58c9df95-f1b3-491b-9152-3b14cdf94c93") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 11:14:59.458768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.458740    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-86.ec2.internal" event="NodeReady"
Sep 10 11:14:59.459176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.458879    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 11:14:59.511777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.511743    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:14:59.544915 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.544884    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:14:59.545059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.544932    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6nm6f"]
Sep 10 11:14:59.545125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.545084    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.550253 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.550224    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 11:14:59.550408 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.550269    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ddwrf\""
Sep 10 11:14:59.551507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.551363    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 11:14:59.551507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.551363    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 11:14:59.555251 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.554482    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-682b2"]
Sep 10 11:14:59.555251 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.554621    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.557353 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.557334    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krm6r\""
Sep 10 11:14:59.557463 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.557385    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 11:14:59.557800 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.557784    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 11:14:59.562962 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.562939    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 11:14:59.570821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.570803    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6nm6f"]
Sep 10 11:14:59.570900 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.570843    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-682b2"]
Sep 10 11:14:59.570952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.570929    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.575086 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.574483    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w7x6j\""
Sep 10 11:14:59.575086 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.574570    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 11:14:59.575657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.575630    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 11:14:59.575885 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.575870    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 11:14:59.628866 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.628843    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwlb\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.628976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.628874    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.628976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.628894    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.628976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.628953    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.628981    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629007    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629114    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a12913c6-6220-42c6-af23-16abe84a67c0-config-volume\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.629328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629148    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12913c6-6220-42c6-af23-16abe84a67c0-tmp-dir\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.629328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629214    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.629328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.629273    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzcc\" (UniqueName: \"kubernetes.io/projected/a12913c6-6220-42c6-af23-16abe84a67c0-kube-api-access-nbzcc\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.657176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.657154    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:14:59.657317 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.657271    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:14:59.657701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.657410    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:14:59.660362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.660336    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 11:14:59.662429 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.662396    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 11:14:59.662525 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.662414    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 11:14:59.669227 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.669210    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9kdff\""
Sep 10 11:14:59.671439 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.671411    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7sc2\""
Sep 10 11:14:59.671532 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.671413    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 11:14:59.730416 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwlb\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730416 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730368    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730416 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730398    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730455    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:59.730543    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:59.730549    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730573    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730591    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:59.730634    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:00.23061271 +0000 UTC m=+35.034680053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:14:59.730666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730658    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:59.730555    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a12913c6-6220-42c6-af23-16abe84a67c0-config-volume\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730720    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12913c6-6220-42c6-af23-16abe84a67c0-tmp-dir\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:14:59.730739    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:00.23072236 +0000 UTC m=+35.034789693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730771    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8fvx\" (UniqueName: \"kubernetes.io/projected/41ae446d-970e-4fc8-8084-cff298f430c9-kube-api-access-h8fvx\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730803    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730825    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41ae446d-970e-4fc8-8084-cff298f430c9-cert\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.730904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730850    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.731455 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730998    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a12913c6-6220-42c6-af23-16abe84a67c0-tmp-dir\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.731455 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.730988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzcc\" (UniqueName: \"kubernetes.io/projected/a12913c6-6220-42c6-af23-16abe84a67c0-kube-api-access-nbzcc\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.731568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.731453    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.731690 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.731672    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.731749 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.731720    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a12913c6-6220-42c6-af23-16abe84a67c0-config-volume\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.731859 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.731834    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.735728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.735702    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.735854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.735702    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.752836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.752811    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzcc\" (UniqueName: \"kubernetes.io/projected/a12913c6-6220-42c6-af23-16abe84a67c0-kube-api-access-nbzcc\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:14:59.753647 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.753624    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.760668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.760645    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwlb\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:14:59.832195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.832165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8fvx\" (UniqueName: \"kubernetes.io/projected/41ae446d-970e-4fc8-8084-cff298f430c9-kube-api-access-h8fvx\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.832355 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.832202    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41ae446d-970e-4fc8-8084-cff298f430c9-cert\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.834560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.834537    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/41ae446d-970e-4fc8-8084-cff298f430c9-cert\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.846405 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.846385    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8fvx\" (UniqueName: \"kubernetes.io/projected/41ae446d-970e-4fc8-8084-cff298f430c9-kube-api-access-h8fvx\") pod \"ingress-canary-682b2\" (UID: \"41ae446d-970e-4fc8-8084-cff298f430c9\") " pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:14:59.883480 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:14:59.883444    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-682b2"
Sep 10 11:15:00.234983 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:00.234947    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:00.235174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:00.235000    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:00.235174 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:00.235107    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:00.235174 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:00.235110    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:00.235174 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:00.235121    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:00.235375 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:00.235179    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:01.235157575 +0000 UTC m=+36.039224928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:00.235375 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:00.235213    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:01.235196383 +0000 UTC m=+36.039263740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:01.244569 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.244529    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.244627    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:01.244696    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:01.244713    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:01.244720    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:01.244776    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:03.24475768 +0000 UTC m=+38.048825020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:01.245094 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:01.244793    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:03.244783754 +0000 UTC m=+38.048851080 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:01.663698 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.663469    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-682b2"]
Sep 10 11:15:01.668313 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:01.668265    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ae446d_970e_4fc8_8084_cff298f430c9.slice/crio-951e647156609b11f0db7fb83f830f9712e682f02276f52f42e6791b3bf238aa WatchSource:0}: Error finding container 951e647156609b11f0db7fb83f830f9712e682f02276f52f42e6791b3bf238aa: Status 404 returned error can't find the container with id 951e647156609b11f0db7fb83f830f9712e682f02276f52f42e6791b3bf238aa
Sep 10 11:15:01.943204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.943177    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="8de398a9a9ab06dabcad56a835b21763a6371b1b0b14d68ec56ba261c98cd55c" exitCode=0
Sep 10 11:15:01.943359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.943254    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"8de398a9a9ab06dabcad56a835b21763a6371b1b0b14d68ec56ba261c98cd55c"}
Sep 10 11:15:01.944473 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:01.944454    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-682b2" event={"ID":"41ae446d-970e-4fc8-8084-cff298f430c9","Type":"ContainerStarted","Data":"951e647156609b11f0db7fb83f830f9712e682f02276f52f42e6791b3bf238aa"}
Sep 10 11:15:02.950946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:02.950899    2585 generic.go:356] "Generic (PLEG): container finished" podID="7cf2c07f-5632-49fc-85df-587b69a18319" containerID="085da47c0cd2f0906a76828c34175847f0d09afe5ed4e45a3b6f6fd79477a361" exitCode=0
Sep 10 11:15:02.950946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:02.950933    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerDied","Data":"085da47c0cd2f0906a76828c34175847f0d09afe5ed4e45a3b6f6fd79477a361"}
Sep 10 11:15:03.260629 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:03.260596    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:03.260779 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:03.260649    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:03.260825 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:03.260773    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:03.260875 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:03.260776    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:03.260912 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:03.260877    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:03.260912 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:03.260878    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:07.260858899 +0000 UTC m=+42.064926245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:03.260991 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:03.260919    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:07.260909544 +0000 UTC m=+42.064976877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:03.957782 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:03.957750    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" event={"ID":"7cf2c07f-5632-49fc-85df-587b69a18319","Type":"ContainerStarted","Data":"3ddc12f9901e8045369d50c43175d58718f1e8ab254e66a10bce2661e1260d18"}
Sep 10 11:15:05.963990 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:05.963955    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-682b2" event={"ID":"41ae446d-970e-4fc8-8084-cff298f430c9","Type":"ContainerStarted","Data":"419182fecde2757e4872cd7b1a9b931d84ab78ed746ac4f2728171086fc1c90c"}
Sep 10 11:15:05.990283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:05.990231    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wjhd7" podStartSLOduration=5.376135099 podStartE2EDuration="39.9902164s" totalImagesPullingTime="34.614081301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-10 11:15:03.994860157 +0000 UTC m=+38.798927504" watchObservedRunningTime="2026-09-10 11:15:05.9902164 +0000 UTC m=+40.794283748"
Sep 10 11:15:07.290941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:07.290905    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:07.290975    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:07.291064    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:07.291077    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:07.291103    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:07.291113    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:15.291100702 +0000 UTC m=+50.095168034 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:07.291387 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:07.291156    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:15.291138949 +0000 UTC m=+50.095206278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:15.350634 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:15.350588    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:15.350648    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:15.350737    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:15.350740    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:15.350748    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:15.350811    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:31.350792613 +0000 UTC m=+66.154859966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:15.351043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:15.350838    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:15:31.350825766 +0000 UTC m=+66.154893110 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:24.937090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:24.937063    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c4glv"
Sep 10 11:15:24.972839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:24.972792    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-682b2" podStartSLOduration=22.600529468 podStartE2EDuration="25.972778861s" totalImagesPullingTime="3.372249393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:05.989909639 +0000 UTC m=+40.793976987" watchObservedRunningTime="2026-09-10 11:15:24.972778861 +0000 UTC m=+59.776846244"
Sep 10 11:15:28.857289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.857251    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"]
Sep 10 11:15:28.860077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.860037    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:28.866872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.866854    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 11:15:28.867058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.867046    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 11:15:28.867197 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.867179    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 11:15:28.867562 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.867549    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9v4s5\""
Sep 10 11:15:28.867779 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.867761    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:15:28.884316 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.884274    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"]
Sep 10 11:15:28.943149 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.943123    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b445af6-616b-4a61-aa56-e540042678a4-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:28.943262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.943235    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b445af6-616b-4a61-aa56-e540042678a4-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:28.943351 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:28.943275    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpvmw\" (UniqueName: \"kubernetes.io/projected/9b445af6-616b-4a61-aa56-e540042678a4-kube-api-access-gpvmw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.044386 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.044361    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b445af6-616b-4a61-aa56-e540042678a4-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.044504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.044436    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b445af6-616b-4a61-aa56-e540042678a4-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.044504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.044456    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpvmw\" (UniqueName: \"kubernetes.io/projected/9b445af6-616b-4a61-aa56-e540042678a4-kube-api-access-gpvmw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.044999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.044978    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9b445af6-616b-4a61-aa56-e540042678a4-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.047409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.047388    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9b445af6-616b-4a61-aa56-e540042678a4-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.058140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.058119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpvmw\" (UniqueName: \"kubernetes.io/projected/9b445af6-616b-4a61-aa56-e540042678a4-kube-api-access-gpvmw\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pnftn\" (UID: \"9b445af6-616b-4a61-aa56-e540042678a4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.168669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.168645    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"
Sep 10 11:15:29.447982 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:29.447896    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn"]
Sep 10 11:15:29.452058 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:29.452012    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b445af6_616b_4a61_aa56_e540042678a4.slice/crio-67a8fa50667c0542f28c68c83ead89540c9b0cef644abf852386425c7a29d1f7 WatchSource:0}: Error finding container 67a8fa50667c0542f28c68c83ead89540c9b0cef644abf852386425c7a29d1f7: Status 404 returned error can't find the container with id 67a8fa50667c0542f28c68c83ead89540c9b0cef644abf852386425c7a29d1f7
Sep 10 11:15:30.019953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.019913    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn" event={"ID":"9b445af6-616b-4a61-aa56-e540042678a4","Type":"ContainerStarted","Data":"67a8fa50667c0542f28c68c83ead89540c9b0cef644abf852386425c7a29d1f7"}
Sep 10 11:15:30.355678 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.355585    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:15:30.358200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.358175    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 11:15:30.369275 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.369250    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60beda01-3d15-4bc8-9682-d5d86b363440-original-pull-secret\") pod \"global-pull-secret-syncer-bzrjs\" (UID: \"60beda01-3d15-4bc8-9682-d5d86b363440\") " pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:15:30.456684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.456647    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:15:30.459265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.459243    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 11:15:30.467452 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:30.467430    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 11:15:30.467564 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:30.467507    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs podName:755bead5-09ff-4756-a084-0f327c88a9e1 nodeName:}" failed. No retries permitted until 2026-09-10 11:16:34.46748602 +0000 UTC m=+129.271553360 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs") pod "network-metrics-daemon-wsmgl" (UID: "755bead5-09ff-4756-a084-0f327c88a9e1") : secret "metrics-daemon-secret" not found
Sep 10 11:15:30.557765 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.557732    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:15:30.560394 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.560372    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 11:15:30.568519 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.568483    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzrjs"
Sep 10 11:15:30.570687 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.570671    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 11:15:30.581651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.581617    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59fbf\" (UniqueName: \"kubernetes.io/projected/58c9df95-f1b3-491b-9152-3b14cdf94c93-kube-api-access-59fbf\") pod \"network-check-target-jktjz\" (UID: \"58c9df95-f1b3-491b-9152-3b14cdf94c93\") " pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:15:30.853929 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.853887    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzrjs"]
Sep 10 11:15:30.858414 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:30.858387    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60beda01_3d15_4bc8_9682_d5d86b363440.slice/crio-1438228f004077f533e2326f54cbba6a8180b859ed1115aeef9ded0fd27f6255 WatchSource:0}: Error finding container 1438228f004077f533e2326f54cbba6a8180b859ed1115aeef9ded0fd27f6255: Status 404 returned error can't find the container with id 1438228f004077f533e2326f54cbba6a8180b859ed1115aeef9ded0fd27f6255
Sep 10 11:15:30.878416 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.878363    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7sc2\""
Sep 10 11:15:30.885737 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:30.885719    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:15:31.023707 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.023677    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"]
Sep 10 11:15:31.049371 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.049344    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"]
Sep 10 11:15:31.049523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.049385    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bzrjs" event={"ID":"60beda01-3d15-4bc8-9682-d5d86b363440","Type":"ContainerStarted","Data":"1438228f004077f533e2326f54cbba6a8180b859ed1115aeef9ded0fd27f6255"}
Sep 10 11:15:31.049523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.049459    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.052545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.052525    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 11:15:31.052669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.052587    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:15:31.053241 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.053220    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6lcpm\""
Sep 10 11:15:31.053367 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.053244    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 11:15:31.053762 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.053741    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 11:15:31.060325 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.060288    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a24adcb-7e33-441f-adac-89488c948515-serving-cert\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.060435 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.060362    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w2dd\" (UniqueName: \"kubernetes.io/projected/7a24adcb-7e33-441f-adac-89488c948515-kube-api-access-6w2dd\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.060435 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.060409    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a24adcb-7e33-441f-adac-89488c948515-config\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.155176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.155108    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jktjz"]
Sep 10 11:15:31.160792 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.160759    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a24adcb-7e33-441f-adac-89488c948515-serving-cert\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.160792 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.160791    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2dd\" (UniqueName: \"kubernetes.io/projected/7a24adcb-7e33-441f-adac-89488c948515-kube-api-access-6w2dd\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.160978 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.160825    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a24adcb-7e33-441f-adac-89488c948515-config\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.161728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.161697    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a24adcb-7e33-441f-adac-89488c948515-config\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.163119 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.163097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a24adcb-7e33-441f-adac-89488c948515-serving-cert\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.169830 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.169801    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w2dd\" (UniqueName: \"kubernetes.io/projected/7a24adcb-7e33-441f-adac-89488c948515-kube-api-access-6w2dd\") pod \"service-ca-operator-859f94c4b-phkb9\" (UID: \"7a24adcb-7e33-441f-adac-89488c948515\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.359498 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.359457    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"
Sep 10 11:15:31.362896 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.362856    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:15:31.363029 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.362919    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:15:31.363029 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:31.362989    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 11:15:31.363138 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:31.363045    2585 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 11:15:31.363138 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:31.363059    2585 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f6fd9c568-l64w2: secret "image-registry-tls" not found
Sep 10 11:15:31.363138 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:31.363074    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls podName:a12913c6-6220-42c6-af23-16abe84a67c0 nodeName:}" failed. No retries permitted until 2026-09-10 11:16:03.363052583 +0000 UTC m=+98.167119912 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls") pod "dns-default-6nm6f" (UID: "a12913c6-6220-42c6-af23-16abe84a67c0") : secret "dns-default-metrics-tls" not found
Sep 10 11:15:31.363138 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:31.363105    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls podName:74fd4771-e64a-4e78-a447-83f8184734ef nodeName:}" failed. No retries permitted until 2026-09-10 11:16:03.363090248 +0000 UTC m=+98.167157597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls") pod "image-registry-6f6fd9c568-l64w2" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef") : secret "image-registry-tls" not found
Sep 10 11:15:31.627951 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:31.627902    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c9df95_f1b3_491b_9152_3b14cdf94c93.slice/crio-eedb6cf560a8758a2f3e8f7b4e6b6fe428d80b45ae456bde8a40a9fa5ebeebe0 WatchSource:0}: Error finding container eedb6cf560a8758a2f3e8f7b4e6b6fe428d80b45ae456bde8a40a9fa5ebeebe0: Status 404 returned error can't find the container with id eedb6cf560a8758a2f3e8f7b4e6b6fe428d80b45ae456bde8a40a9fa5ebeebe0
Sep 10 11:15:31.894615 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:31.894594    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9"]
Sep 10 11:15:31.897742 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:31.897719    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a24adcb_7e33_441f_adac_89488c948515.slice/crio-e638a12c288c921d699bd1288d999fa2873b2e331cbe57c056374ea77e1c28b4 WatchSource:0}: Error finding container e638a12c288c921d699bd1288d999fa2873b2e331cbe57c056374ea77e1c28b4: Status 404 returned error can't find the container with id e638a12c288c921d699bd1288d999fa2873b2e331cbe57c056374ea77e1c28b4
Sep 10 11:15:32.028964 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.028929    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn" event={"ID":"9b445af6-616b-4a61-aa56-e540042678a4","Type":"ContainerStarted","Data":"9c8562d163198c517634a98ecc85d1bd1341509512b429d1e9323d11153bb0f2"}
Sep 10 11:15:32.031400 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.030215    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jktjz" event={"ID":"58c9df95-f1b3-491b-9152-3b14cdf94c93","Type":"ContainerStarted","Data":"eedb6cf560a8758a2f3e8f7b4e6b6fe428d80b45ae456bde8a40a9fa5ebeebe0"}
Sep 10 11:15:32.031931 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.031904    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9" event={"ID":"7a24adcb-7e33-441f-adac-89488c948515","Type":"ContainerStarted","Data":"e638a12c288c921d699bd1288d999fa2873b2e331cbe57c056374ea77e1c28b4"}
Sep 10 11:15:32.047440 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.047419    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9qppc"]
Sep 10 11:15:32.065315 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.065258    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn" podStartSLOduration=1.688854843 podStartE2EDuration="4.065245644s" totalImagesPullingTime="2.376390801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:32.062589746 +0000 UTC m=+66.866657093" watchObservedRunningTime="2026-09-10 11:15:32.065245644 +0000 UTC m=+66.869312992"
Sep 10 11:15:32.067579 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.067562    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.074619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.074599    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 11:15:32.074959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.074937    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 11:15:32.075099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.075063    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tf5wn\""
Sep 10 11:15:32.075257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.075237    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9qppc"]
Sep 10 11:15:32.075545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.075529    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 11:15:32.075845 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.075831    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 11:15:32.170924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.170887    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ff60e0-f7d0-416f-9414-4e9822ea06a8-data-volume\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.171085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.170953    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vn6p\" (UniqueName: \"kubernetes.io/projected/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-api-access-9vn6p\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.171085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.171045    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ff60e0-f7d0-416f-9414-4e9822ea06a8-crio-socket\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.171203 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.171097    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.171203 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.171128    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271560    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271601    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ff60e0-f7d0-416f-9414-4e9822ea06a8-data-volume\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271639    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vn6p\" (UniqueName: \"kubernetes.io/projected/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-api-access-9vn6p\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271834 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.271700    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:15:32.271834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271721    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ff60e0-f7d0-416f-9414-4e9822ea06a8-crio-socket\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.271834 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.271788    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:32.771767893 +0000 UTC m=+67.575835235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : configmap "kube-rbac-proxy" not found
Sep 10 11:15:32.272089 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.271867    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:32.272089 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.271918    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:32.771903291 +0000 UTC m=+67.575970640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:32.272089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.271968    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ff60e0-f7d0-416f-9414-4e9822ea06a8-crio-socket\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.272089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.272015    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ff60e0-f7d0-416f-9414-4e9822ea06a8-data-volume\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.286537 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.286511    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vn6p\" (UniqueName: \"kubernetes.io/projected/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-api-access-9vn6p\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.774530 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.774494    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.774726 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:32.774546    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:32.774726 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.774653    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:32.774726 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.774687    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:15:32.774894 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.774723    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:33.774705247 +0000 UTC m=+68.578772586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:32.774894 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:32.774757    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:33.774747251 +0000 UTC m=+68.578814582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : configmap "kube-rbac-proxy" not found
Sep 10 11:15:33.784361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:33.784317    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:33.784902 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:33.784382    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:33.784902 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:33.784429    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:33.784902 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:33.784505    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:15:33.784902 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:33.784509    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:35.784488082 +0000 UTC m=+70.588555411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:33.784902 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:33.784569    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:35.784553025 +0000 UTC m=+70.588620351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : configmap "kube-rbac-proxy" not found
Sep 10 11:15:34.148747 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.148602    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qv7xl_ded2fd1b-1a87-4842-a072-82ee5c3fc4a2/dns-node-resolver/0.log"
Sep 10 11:15:34.706124 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.706085    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"]
Sep 10 11:15:34.709058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.709035    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"
Sep 10 11:15:34.712439 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.712420    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 11:15:34.713034 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.713011    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-pkhxm\""
Sep 10 11:15:34.713153 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.713013    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 11:15:34.725239 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.725210    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"]
Sep 10 11:15:34.793349 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.793319    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bcdk\" (UniqueName: \"kubernetes.io/projected/fd08c3f5-61f0-4253-988a-3f5fa3ef80b8-kube-api-access-6bcdk\") pod \"migrator-fdbb8fdff-ps9b7\" (UID: \"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"
Sep 10 11:15:34.894778 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.894741    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcdk\" (UniqueName: \"kubernetes.io/projected/fd08c3f5-61f0-4253-988a-3f5fa3ef80b8-kube-api-access-6bcdk\") pod \"migrator-fdbb8fdff-ps9b7\" (UID: \"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"
Sep 10 11:15:34.904566 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.904538    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bcdk\" (UniqueName: \"kubernetes.io/projected/fd08c3f5-61f0-4253-988a-3f5fa3ef80b8-kube-api-access-6bcdk\") pod \"migrator-fdbb8fdff-ps9b7\" (UID: \"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"
Sep 10 11:15:34.947663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:34.947636    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tf7ph_8967943b-d3b6-444f-a079-a06ed65e35c9/node-ca/0.log"
Sep 10 11:15:35.020553 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:35.020468    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"
Sep 10 11:15:35.550130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:35.550076    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-682b2_41ae446d-970e-4fc8-8084-cff298f430c9/serve-healthcheck-canary/0.log"
Sep 10 11:15:35.801617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:35.801534    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:35.801617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:35.801587    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:35.802064 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:35.801698    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:15:35.802064 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:35.801762    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:39.801742232 +0000 UTC m=+74.605809564 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : configmap "kube-rbac-proxy" not found
Sep 10 11:15:35.802064 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:35.801699    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:35.802064 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:35.801835    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:39.801822136 +0000 UTC m=+74.605889468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:37.054226 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.054147    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9" event={"ID":"7a24adcb-7e33-441f-adac-89488c948515","Type":"ContainerStarted","Data":"7b9340cf1abf065497f528a043875815169a5b1d2231b80910fad20606f49776"}
Sep 10 11:15:37.055813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.055786    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bzrjs" event={"ID":"60beda01-3d15-4bc8-9682-d5d86b363440","Type":"ContainerStarted","Data":"ba393e8c97014d75c9dcfee23d11c143b2c65ab6175ccf39a8331d83780ec1e0"}
Sep 10 11:15:37.057687 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.057545    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jktjz" event={"ID":"58c9df95-f1b3-491b-9152-3b14cdf94c93","Type":"ContainerStarted","Data":"761768efa12d657d6871a35dca6ff1ddd425b0d93b47cf16fdfd67e947619d4f"}
Sep 10 11:15:37.057835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.057810    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:15:37.059365 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.059294    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7"]
Sep 10 11:15:37.061994 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:15:37.061973    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd08c3f5_61f0_4253_988a_3f5fa3ef80b8.slice/crio-3b574f0a34b70dc59e459a0e01b6d93c2052d3451b4cdfa2c5bdd03c258351e5 WatchSource:0}: Error finding container 3b574f0a34b70dc59e459a0e01b6d93c2052d3451b4cdfa2c5bdd03c258351e5: Status 404 returned error can't find the container with id 3b574f0a34b70dc59e459a0e01b6d93c2052d3451b4cdfa2c5bdd03c258351e5
Sep 10 11:15:37.082687 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.082649    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9" podStartSLOduration=2.206742029 podStartE2EDuration="7.082636732s" totalImagesPullingTime="4.875894703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:37.082047115 +0000 UTC m=+71.886114463" watchObservedRunningTime="2026-09-10 11:15:37.082636732 +0000 UTC m=+71.886704080"
Sep 10 11:15:37.106129 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:37.106094    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bzrjs" podStartSLOduration=65.175600746 podStartE2EDuration="1m11.106082059s" totalImagesPullingTime="5.930481313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:37.105316986 +0000 UTC m=+71.909384336" watchObservedRunningTime="2026-09-10 11:15:37.106082059 +0000 UTC m=+71.910149384"
Sep 10 11:15:38.062180 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:38.062144    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7" event={"ID":"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8","Type":"ContainerStarted","Data":"3b574f0a34b70dc59e459a0e01b6d93c2052d3451b4cdfa2c5bdd03c258351e5"}
Sep 10 11:15:39.066920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.066890    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7" event={"ID":"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8","Type":"ContainerStarted","Data":"e0d2e879923c2d6ed686d74fc7301b06cfa0bd6af427a1135cb004a1f17279b7"}
Sep 10 11:15:39.067270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.066938    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7" event={"ID":"fd08c3f5-61f0-4253-988a-3f5fa3ef80b8","Type":"ContainerStarted","Data":"14e51d8890552e63d17d34f6ee0f7290b0943f025a1804f11e2d7b4b8779fc22"}
Sep 10 11:15:39.086777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.086732    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-ps9b7" podStartSLOduration=3.673627029 podStartE2EDuration="5.086716665s" totalImagesPullingTime="1.413089636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:15:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:39.085767268 +0000 UTC m=+73.889834627" watchObservedRunningTime="2026-09-10 11:15:39.086716665 +0000 UTC m=+73.890784013"
Sep 10 11:15:39.087256 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.087235    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jktjz" podStartSLOduration=67.933334804 podStartE2EDuration="1m13.087226883s" totalImagesPullingTime="5.153892079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:15:37.129166535 +0000 UTC m=+71.933233883" watchObservedRunningTime="2026-09-10 11:15:39.087226883 +0000 UTC m=+73.891294232"
Sep 10 11:15:39.835436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.835395    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:39.835436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:39.835441    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:39.835682 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:39.835562    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 11:15:39.835682 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:39.835578    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:39.835682 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:39.835628    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:47.835610422 +0000 UTC m=+82.639677765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : configmap "kube-rbac-proxy" not found
Sep 10 11:15:39.835682 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:39.835646    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:15:47.835637223 +0000 UTC m=+82.639704549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:47.894634 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:47.894597    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:47.894634 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:47.894637    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:15:47.895037 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:47.894757    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:47.895037 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:15:47.894828    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls podName:57ff60e0-f7d0-416f-9414-4e9822ea06a8 nodeName:}" failed. No retries permitted until 2026-09-10 11:16:03.894813414 +0000 UTC m=+98.698880744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9qppc" (UID: "57ff60e0-f7d0-416f-9414-4e9822ea06a8") : secret "insights-runtime-extractor-tls" not found
Sep 10 11:15:47.895196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:15:47.895175    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ff60e0-f7d0-416f-9414-4e9822ea06a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:16:01.272764 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.272732    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"]
Sep 10 11:16:01.276758 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.276738    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.282810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.282788    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 11:16:01.282907 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.282788    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 11:16:01.283263 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.283248    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 11:16:01.283313 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.283276    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 11:16:01.334312 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.334285    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"]
Sep 10 11:16:01.393732 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.393706    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bbbf773c-a743-47ae-8be8-057c08446661-klusterlet-config\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.393865 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.393746    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vpcc\" (UniqueName: \"kubernetes.io/projected/bbbf773c-a743-47ae-8be8-057c08446661-kube-api-access-5vpcc\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.393865 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.393787    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbbf773c-a743-47ae-8be8-057c08446661-tmp\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.494620 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.494592    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bbbf773c-a743-47ae-8be8-057c08446661-klusterlet-config\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.494773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.494631    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vpcc\" (UniqueName: \"kubernetes.io/projected/bbbf773c-a743-47ae-8be8-057c08446661-kube-api-access-5vpcc\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.494819 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.494770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbbf773c-a743-47ae-8be8-057c08446661-tmp\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.495117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.495097    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbbf773c-a743-47ae-8be8-057c08446661-tmp\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.497143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.497118    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/bbbf773c-a743-47ae-8be8-057c08446661-klusterlet-config\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.521518 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.521489    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vpcc\" (UniqueName: \"kubernetes.io/projected/bbbf773c-a743-47ae-8be8-057c08446661-kube-api-access-5vpcc\") pod \"klusterlet-addon-workmgr-65966d47bf-kr62c\" (UID: \"bbbf773c-a743-47ae-8be8-057c08446661\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.589052 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.588992    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:01.877965 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:01.877938    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"]
Sep 10 11:16:01.880244 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:01.880216    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbbf773c_a743_47ae_8be8_057c08446661.slice/crio-22ffbdfdddada5833482316edb1400ea8c06d7da3cf639e95326b2050c71021f WatchSource:0}: Error finding container 22ffbdfdddada5833482316edb1400ea8c06d7da3cf639e95326b2050c71021f: Status 404 returned error can't find the container with id 22ffbdfdddada5833482316edb1400ea8c06d7da3cf639e95326b2050c71021f
Sep 10 11:16:02.137629 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:02.137546    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c" event={"ID":"bbbf773c-a743-47ae-8be8-057c08446661","Type":"ContainerStarted","Data":"22ffbdfdddada5833482316edb1400ea8c06d7da3cf639e95326b2050c71021f"}
Sep 10 11:16:03.275992 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.275955    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"]
Sep 10 11:16:03.279384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.279348    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:03.283223 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:03.283192    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-operator-admission-webhook-78f599db79-vm4wr\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="5ee02f86-2fc0-4e89-a07b-27f2e7d18be3" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:03.283445 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:03.283223    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-dockercfg-vk94x\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vk94x\"" type="*v1.Secret"
Sep 10 11:16:03.283762 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:03.283732    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-operator-admission-webhook-tls\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" type="*v1.Secret"
Sep 10 11:16:03.297281 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.297212    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"]
Sep 10 11:16:03.412704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.412290    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:03.412704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.412372    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:16:03.412704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.412439    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vm4wr\" (UID: \"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:03.415108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.415081    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a12913c6-6220-42c6-af23-16abe84a67c0-metrics-tls\") pod \"dns-default-6nm6f\" (UID: \"a12913c6-6220-42c6-af23-16abe84a67c0\") " pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:16:03.415245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.415173    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"image-registry-6f6fd9c568-l64w2\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") " pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:03.463571 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.463534    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ddwrf\""
Sep 10 11:16:03.469284 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.469259    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krm6r\""
Sep 10 11:16:03.472013 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.471975    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:03.477283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.477257    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:16:03.513046 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.513022    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vm4wr\" (UID: \"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:03.916193 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.916154    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:16:03.918941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:03.918914    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ff60e0-f7d0-416f-9414-4e9822ea06a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9qppc\" (UID: \"57ff60e0-f7d0-416f-9414-4e9822ea06a8\") " pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:16:04.177269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.177186    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9qppc"
Sep 10 11:16:04.351507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.351477    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6nm6f"]
Sep 10 11:16:04.355678 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:04.355654    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda12913c6_6220_42c6_af23_16abe84a67c0.slice/crio-b267b30c682793c467a669877eb30d113438a8ffdca8175ace76c8046d7add14 WatchSource:0}: Error finding container b267b30c682793c467a669877eb30d113438a8ffdca8175ace76c8046d7add14: Status 404 returned error can't find the container with id b267b30c682793c467a669877eb30d113438a8ffdca8175ace76c8046d7add14
Sep 10 11:16:04.361396 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.361372    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:16:04.478554 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.478471    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9qppc"]
Sep 10 11:16:04.513746 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:04.513711    2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:16:04.513922 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:04.513808    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates podName:5ee02f86-2fc0-4e89-a07b-27f2e7d18be3 nodeName:}" failed. No retries permitted until 2026-09-10 11:16:05.013785504 +0000 UTC m=+99.817852838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-vm4wr" (UID: "5ee02f86-2fc0-4e89-a07b-27f2e7d18be3") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:16:04.541413 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.541383    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 11:16:04.576290 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:04.576257    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vk94x\""
Sep 10 11:16:05.025488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:05.025444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vm4wr\" (UID: \"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:05.028200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:05.028176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5ee02f86-2fc0-4e89-a07b-27f2e7d18be3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-vm4wr\" (UID: \"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:05.090371 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:05.090338    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:05.149810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:05.149775    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nm6f" event={"ID":"a12913c6-6220-42c6-af23-16abe84a67c0","Type":"ContainerStarted","Data":"b267b30c682793c467a669877eb30d113438a8ffdca8175ace76c8046d7add14"}
Sep 10 11:16:05.523360 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:05.523323    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ff60e0_f7d0_416f_9414_4e9822ea06a8.slice/crio-0159861f223d26cc103924bdb604c61ec3b8144844d455b5fb1f183012eb9104 WatchSource:0}: Error finding container 0159861f223d26cc103924bdb604c61ec3b8144844d455b5fb1f183012eb9104: Status 404 returned error can't find the container with id 0159861f223d26cc103924bdb604c61ec3b8144844d455b5fb1f183012eb9104
Sep 10 11:16:05.524042 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:05.524012    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fd4771_e64a_4e78_a447_83f8184734ef.slice/crio-39b542914ac6f0ae71cb34f80c773f9b9bdbbc1a9205d78d37de47c613ca798d WatchSource:0}: Error finding container 39b542914ac6f0ae71cb34f80c773f9b9bdbbc1a9205d78d37de47c613ca798d: Status 404 returned error can't find the container with id 39b542914ac6f0ae71cb34f80c773f9b9bdbbc1a9205d78d37de47c613ca798d
Sep 10 11:16:05.796937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:05.796855    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"]
Sep 10 11:16:05.799710 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:05.799685    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee02f86_2fc0_4e89_a07b_27f2e7d18be3.slice/crio-55de336432ad63d3bdd8560c94bbe8f0728cc28b56d05e2b5fb1d040c689e054 WatchSource:0}: Error finding container 55de336432ad63d3bdd8560c94bbe8f0728cc28b56d05e2b5fb1d040c689e054: Status 404 returned error can't find the container with id 55de336432ad63d3bdd8560c94bbe8f0728cc28b56d05e2b5fb1d040c689e054
Sep 10 11:16:06.154668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.154593    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr" event={"ID":"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3","Type":"ContainerStarted","Data":"55de336432ad63d3bdd8560c94bbe8f0728cc28b56d05e2b5fb1d040c689e054"}
Sep 10 11:16:06.156503 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.156473    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" event={"ID":"74fd4771-e64a-4e78-a447-83f8184734ef","Type":"ContainerStarted","Data":"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"}
Sep 10 11:16:06.156644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.156523    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" event={"ID":"74fd4771-e64a-4e78-a447-83f8184734ef","Type":"ContainerStarted","Data":"39b542914ac6f0ae71cb34f80c773f9b9bdbbc1a9205d78d37de47c613ca798d"}
Sep 10 11:16:06.156762 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.156707    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:06.158509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.158148    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c" event={"ID":"bbbf773c-a743-47ae-8be8-057c08446661","Type":"ContainerStarted","Data":"fb2ef1b297767c59b8aedc25ff7bfa9104336d3d39dd52e6a5164c1a716fd153"}
Sep 10 11:16:06.158509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.158333    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:06.160028 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.160002    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qppc" event={"ID":"57ff60e0-f7d0-416f-9414-4e9822ea06a8","Type":"ContainerStarted","Data":"c46a81d12e18fb8bd54db183e05e40e55989ca5c29e8396e29225fec6d2619f8"}
Sep 10 11:16:06.160130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.160082    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c"
Sep 10 11:16:06.160130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.160100    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qppc" event={"ID":"57ff60e0-f7d0-416f-9414-4e9822ea06a8","Type":"ContainerStarted","Data":"0159861f223d26cc103924bdb604c61ec3b8144844d455b5fb1f183012eb9104"}
Sep 10 11:16:06.192804 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.192753    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" podStartSLOduration=100.192737357 podStartE2EDuration="1m40.192737357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:06.192274835 +0000 UTC m=+100.996342183" watchObservedRunningTime="2026-09-10 11:16:06.192737357 +0000 UTC m=+100.996804705"
Sep 10 11:16:06.217738 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:06.217668    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65966d47bf-kr62c" podStartSLOduration=1.274292411 podStartE2EDuration="5.217646415s" totalImagesPullingTime="3.943354004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:06.217367903 +0000 UTC m=+101.021435253" watchObservedRunningTime="2026-09-10 11:16:06.217646415 +0000 UTC m=+101.021713766"
Sep 10 11:16:07.165902 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:07.165863    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qppc" event={"ID":"57ff60e0-f7d0-416f-9414-4e9822ea06a8","Type":"ContainerStarted","Data":"255fa1497a60eb69ca0fb627f0d00283ce7ea698d7aaddfe9b18ffdaca5fb747"}
Sep 10 11:16:08.065421 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:08.065179    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jktjz"
Sep 10 11:16:08.176937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:08.176900    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nm6f" event={"ID":"a12913c6-6220-42c6-af23-16abe84a67c0","Type":"ContainerStarted","Data":"dcb29b8769995921fa20dd19cd5982982a47a9208c682865cc9f658785a42d32"}
Sep 10 11:16:09.186049 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:09.186015    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6nm6f" event={"ID":"a12913c6-6220-42c6-af23-16abe84a67c0","Type":"ContainerStarted","Data":"36fe264885887db2e3d45bb8094fd6a5c0367602f2a6f646ac9469b7affb982d"}
Sep 10 11:16:09.186389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:09.186151    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:16:10.191267 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:10.191232    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9qppc" event={"ID":"57ff60e0-f7d0-416f-9414-4e9822ea06a8","Type":"ContainerStarted","Data":"c62056b5b095f211486961d56eaa1f6e4880a0f5e2534f0a36e44ae64fd43798"}
Sep 10 11:16:10.245121 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:10.245071    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6nm6f" podStartSLOduration=67.711440956 podStartE2EDuration="1m11.245052788s" totalImagesPullingTime="3.533611832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:09.212239443 +0000 UTC m=+104.016306792" watchObservedRunningTime="2026-09-10 11:16:10.245052788 +0000 UTC m=+105.049120134"
Sep 10 11:16:10.245821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:10.245788    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9qppc" podStartSLOduration=34.893858104 podStartE2EDuration="38.245777236s" totalImagesPullingTime="3.351919132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:15:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:10.244558382 +0000 UTC m=+105.048625731" watchObservedRunningTime="2026-09-10 11:16:10.245777236 +0000 UTC m=+105.049844582"
Sep 10 11:16:11.195622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:11.195585    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr" event={"ID":"5ee02f86-2fc0-4e89-a07b-27f2e7d18be3","Type":"ContainerStarted","Data":"f59fb36072abdad547eb5774af073270b49886479a1b141a8aea5a1a8814502a"}
Sep 10 11:16:12.198934 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:12.198900    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:12.203460 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:12.203436    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr"
Sep 10 11:16:12.230529 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:12.230483    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-vm4wr" podStartSLOduration=4.912801175 podStartE2EDuration="9.230467706s" totalImagesPullingTime="4.317666531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:11.221747778 +0000 UTC m=+106.025815126" watchObservedRunningTime="2026-09-10 11:16:12.230467706 +0000 UTC m=+107.034535053"
Sep 10 11:16:13.280031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.279997    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"]
Sep 10 11:16:13.282594 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.282578    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.287901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.287878    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 11:16:13.288776 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.288620    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 11:16:13.288776 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.288685    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-ff9jv\""
Sep 10 11:16:13.288959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.288936    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 11:16:13.289049 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.288964    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 11:16:13.294913 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.294898    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 11:16:13.300785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.300762    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"]
Sep 10 11:16:13.393340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.393313    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw4pf\" (UniqueName: \"kubernetes.io/projected/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-kube-api-access-tw4pf\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.393451 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.393345    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.393451 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.393361    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.393572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.393474    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.494192 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.494163    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.494340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.494198    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw4pf\" (UniqueName: \"kubernetes.io/projected/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-kube-api-access-tw4pf\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.494340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.494228    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.494340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.494253    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.494970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.494940    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.496577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.496554    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.496676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.496592    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.503968 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.503942    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw4pf\" (UniqueName: \"kubernetes.io/projected/4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2-kube-api-access-tw4pf\") pod \"prometheus-operator-6f4946db56-pkkmr\" (UID: \"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.590869 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.590811    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"
Sep 10 11:16:13.864664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:13.864637    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-pkkmr"]
Sep 10 11:16:13.865482 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:13.865454    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5b7cc8_7d7c_4394_b581_6a8b5d7fbfc2.slice/crio-c29369fa0b36f1f01f2e41f436c70259cb219469c5b01d99f51d417a94602935 WatchSource:0}: Error finding container c29369fa0b36f1f01f2e41f436c70259cb219469c5b01d99f51d417a94602935: Status 404 returned error can't find the container with id c29369fa0b36f1f01f2e41f436c70259cb219469c5b01d99f51d417a94602935
Sep 10 11:16:14.206201 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:14.206168    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr" event={"ID":"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2","Type":"ContainerStarted","Data":"c29369fa0b36f1f01f2e41f436c70259cb219469c5b01d99f51d417a94602935"}
Sep 10 11:16:15.212739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:15.212643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr" event={"ID":"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2","Type":"ContainerStarted","Data":"46571c671c062d71f92c8b9221f6296c81b98e8fb7947a9d8fbf1eeab66f927f"}
Sep 10 11:16:15.212739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:15.212710    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr" event={"ID":"4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2","Type":"ContainerStarted","Data":"96de1cb87050a9829ac79fa27f5e6daa9c970ed6d52af2836d5b99d6b92ac16e"}
Sep 10 11:16:15.236589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:15.236538    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-pkkmr" podStartSLOduration=1.154908729 podStartE2EDuration="2.236522905s" totalImagesPullingTime="1.081614176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:15.23443126 +0000 UTC m=+110.038498619" watchObservedRunningTime="2026-09-10 11:16:15.236522905 +0000 UTC m=+110.040590250"
Sep 10 11:16:17.786388 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.786355    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r9m68"]
Sep 10 11:16:17.789517 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.789501    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.791962 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.791939    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 11:16:17.792081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.791971    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 11:16:17.792328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.792314    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cd9j5\""
Sep 10 11:16:17.792625 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.792608    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 11:16:17.925645 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925610    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9hjb\" (UniqueName: \"kubernetes.io/projected/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-kube-api-access-v9hjb\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925650    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925678    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-wtmp\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925716    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925735    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-textfile\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925764    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-sys\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925842    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925879    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-metrics-client-ca\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:17.925979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:17.925905    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-root\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026547 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026517    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026553    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-metrics-client-ca\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-root\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026591    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9hjb\" (UniqueName: \"kubernetes.io/projected/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-kube-api-access-v9hjb\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026645    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026691    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-wtmp\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026647    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-root\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026772    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-textfile\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:18.026786    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-sys\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:18.026844    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls podName:d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb nodeName:}" failed. No retries permitted until 2026-09-10 11:16:18.526825431 +0000 UTC m=+113.330892773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls") pod "node-exporter-r9m68" (UID: "d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb") : secret "node-exporter-tls" not found
Sep 10 11:16:18.026914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.026866    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-wtmp\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.027318 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.027093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-sys\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.027318 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.027093    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-textfile\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.027318 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.027265    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.027479 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.027388    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-metrics-client-ca\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.029069 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.029048    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.039957 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.039895    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9hjb\" (UniqueName: \"kubernetes.io/projected/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-kube-api-access-v9hjb\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.529641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.529606    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.531898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.531874    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb-node-exporter-tls\") pod \"node-exporter-r9m68\" (UID: \"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb\") " pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.698654 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:18.698620    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9m68"
Sep 10 11:16:18.710530 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:18.710505    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d9ea26_cf99_42a6_81f2_3c8c2c98b8fb.slice/crio-6d4e7774ac2df754fc4a4775998b4528e73188f296b2af21be885ed6cdbbc249 WatchSource:0}: Error finding container 6d4e7774ac2df754fc4a4775998b4528e73188f296b2af21be885ed6cdbbc249: Status 404 returned error can't find the container with id 6d4e7774ac2df754fc4a4775998b4528e73188f296b2af21be885ed6cdbbc249
Sep 10 11:16:19.194333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:19.194285    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6nm6f"
Sep 10 11:16:19.227758 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:19.227725    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9m68" event={"ID":"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb","Type":"ContainerStarted","Data":"6d4e7774ac2df754fc4a4775998b4528e73188f296b2af21be885ed6cdbbc249"}
Sep 10 11:16:20.232658 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.232625    2585 generic.go:356] "Generic (PLEG): container finished" podID="d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb" containerID="b76d691fcb9729153317c36b857993947d35f6e7fed06c2de2234c85ae7c2649" exitCode=0
Sep 10 11:16:20.233025 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.232666    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9m68" event={"ID":"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb","Type":"ContainerDied","Data":"b76d691fcb9729153317c36b857993947d35f6e7fed06c2de2234c85ae7c2649"}
Sep 10 11:16:20.802724 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.802691    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"]
Sep 10 11:16:20.806054 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.806030    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.809186 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.809164    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 11:16:20.809279 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.809248    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-85v7kv9hv8hp3\""
Sep 10 11:16:20.809361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.809345    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-z98kr\""
Sep 10 11:16:20.809409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.809386    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 11:16:20.809616 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.809595    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 11:16:20.810336 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.810322    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 11:16:20.811968 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.811948    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 11:16:20.884511 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.884474    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"]
Sep 10 11:16:20.954714 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954681    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-grpc-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.954858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954725    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44447154-eee3-418c-83b0-3e99e0a2f7dd-metrics-client-ca\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.954858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954759    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.954858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954793    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zpcw\" (UniqueName: \"kubernetes.io/projected/44447154-eee3-418c-83b0-3e99e0a2f7dd-kube-api-access-9zpcw\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.954858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954811    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.954858 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954840    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.955049 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954865    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:20.955049 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:20.954881    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056112    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zpcw\" (UniqueName: \"kubernetes.io/projected/44447154-eee3-418c-83b0-3e99e0a2f7dd-kube-api-access-9zpcw\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056195    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-grpc-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056281    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44447154-eee3-418c-83b0-3e99e0a2f7dd-metrics-client-ca\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.056543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.056341    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.057189 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.057130    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44447154-eee3-418c-83b0-3e99e0a2f7dd-metrics-client-ca\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059061    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059074    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059337    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059574 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059404    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059574 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059411    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.059639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.059626    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/44447154-eee3-418c-83b0-3e99e0a2f7dd-secret-grpc-tls\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.070686 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.070662    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zpcw\" (UniqueName: \"kubernetes.io/projected/44447154-eee3-418c-83b0-3e99e0a2f7dd-kube-api-access-9zpcw\") pod \"thanos-querier-6ddd57f5d8-6scl6\" (UID: \"44447154-eee3-418c-83b0-3e99e0a2f7dd\") " pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.115052 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.115020    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:21.237435 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.237398    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9m68" event={"ID":"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb","Type":"ContainerStarted","Data":"16580b4b50c5f65c4d4684cd33e2f77030d16261366273f84226c22be666606d"}
Sep 10 11:16:21.237775 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.237457    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9m68" event={"ID":"d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb","Type":"ContainerStarted","Data":"1e144be69273da0a6fbae05e09c338a715c1882658bf39b1bf471a4b66853247"}
Sep 10 11:16:21.323688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.323614    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-zvmv9"]
Sep 10 11:16:21.326978 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.326962    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:21.338783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.338718    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r9m68" podStartSLOduration=3.332755875 podStartE2EDuration="4.338703898s" totalImagesPullingTime="943.9586ms" totalInitContainerRuntime="61.989423ms" isStatefulPod=true podCreationTimestamp="2026-09-10 11:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:21.338683217 +0000 UTC m=+116.142750565" watchObservedRunningTime="2026-09-10 11:16:21.338703898 +0000 UTC m=+116.142771246"
Sep 10 11:16:21.339265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.339251    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 11:16:21.339941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.339923    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2b652\""
Sep 10 11:16:21.341924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.341908    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 11:16:21.348892 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.348868    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zvmv9"]
Sep 10 11:16:21.412110 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.412080    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"]
Sep 10 11:16:21.418334 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:21.418312    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44447154_eee3_418c_83b0_3e99e0a2f7dd.slice/crio-0f77e969638511eb12297a1546adb976b2e8063b21b580b1994b2ea4b5e36be7 WatchSource:0}: Error finding container 0f77e969638511eb12297a1546adb976b2e8063b21b580b1994b2ea4b5e36be7: Status 404 returned error can't find the container with id 0f77e969638511eb12297a1546adb976b2e8063b21b580b1994b2ea4b5e36be7
Sep 10 11:16:21.459963 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.459919    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxvb9\" (UniqueName: \"kubernetes.io/projected/defa4256-2d6a-44af-ad9b-510d8ae6c24d-kube-api-access-mxvb9\") pod \"downloads-85d98c998f-zvmv9\" (UID: \"defa4256-2d6a-44af-ad9b-510d8ae6c24d\") " pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:21.560446 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.560417    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxvb9\" (UniqueName: \"kubernetes.io/projected/defa4256-2d6a-44af-ad9b-510d8ae6c24d-kube-api-access-mxvb9\") pod \"downloads-85d98c998f-zvmv9\" (UID: \"defa4256-2d6a-44af-ad9b-510d8ae6c24d\") " pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:21.570668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.570647    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxvb9\" (UniqueName: \"kubernetes.io/projected/defa4256-2d6a-44af-ad9b-510d8ae6c24d-kube-api-access-mxvb9\") pod \"downloads-85d98c998f-zvmv9\" (UID: \"defa4256-2d6a-44af-ad9b-510d8ae6c24d\") " pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:21.635772 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.635695    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:21.916490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:21.916461    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-zvmv9"]
Sep 10 11:16:21.919204 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:21.919176    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddefa4256_2d6a_44af_ad9b_510d8ae6c24d.slice/crio-bf1fb74ab9ec576ee05e178cb7afebd9ec77f368717d2c3e3318a1c324b0024a WatchSource:0}: Error finding container bf1fb74ab9ec576ee05e178cb7afebd9ec77f368717d2c3e3318a1c324b0024a: Status 404 returned error can't find the container with id bf1fb74ab9ec576ee05e178cb7afebd9ec77f368717d2c3e3318a1c324b0024a
Sep 10 11:16:22.242388 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.242280    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zvmv9" event={"ID":"defa4256-2d6a-44af-ad9b-510d8ae6c24d","Type":"ContainerStarted","Data":"bf1fb74ab9ec576ee05e178cb7afebd9ec77f368717d2c3e3318a1c324b0024a"}
Sep 10 11:16:22.243641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.243613    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"0f77e969638511eb12297a1546adb976b2e8063b21b580b1994b2ea4b5e36be7"}
Sep 10 11:16:22.430939 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.430900    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"]
Sep 10 11:16:22.434062 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.434043    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.436412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.436387    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 11:16:22.436577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.436560    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hl7r9\""
Sep 10 11:16:22.446806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.446777    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"]
Sep 10 11:16:22.570245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.570162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld8dt\" (UniqueName: \"kubernetes.io/projected/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-kube-api-access-ld8dt\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.570419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.570339    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.671379 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.671340    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.671573 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.671442    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld8dt\" (UniqueName: \"kubernetes.io/projected/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-kube-api-access-ld8dt\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.676998 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.676943    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.686800 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.686771    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld8dt\" (UniqueName: \"kubernetes.io/projected/d4d70ccf-8fe4-413c-ae67-bbcc066f59b1-kube-api-access-ld8dt\") pod \"monitoring-plugin-85bbbbbf4-zrhzc\" (UID: \"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:22.746928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:22.746896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:23.478195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.478140    2585 patch_prober.go:28] interesting pod/image-registry-6f6fd9c568-l64w2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 11:16:23.478558 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.478222    2585 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 11:16:23.602744 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.602235    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"]
Sep 10 11:16:23.974212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.974174    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:16:23.978952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.978913    2585 patch_prober.go:28] interesting pod/image-registry-6f6fd9c568-l64w2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 11:16:23.979088 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:23.978962    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 11:16:24.160160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.160125    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:16:24.164956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.164439    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.170280 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.170254    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 11:16:24.170652 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.170630    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 11:16:24.171097 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171062    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ng9t5\""
Sep 10 11:16:24.171411 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171393    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171589    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.172077    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171655    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171678    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171689    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171722    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171732    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171830    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171871    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eovdk7i5quho3\""
Sep 10 11:16:24.172436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.171598    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 11:16:24.181330 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.180666    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 11:16:24.181951 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.181772    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 11:16:24.185296 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185360    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185391    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185511 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185416    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185511 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185460    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185511 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185483    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185511 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185505    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185580    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185635    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185675    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185840 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185700    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185840 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185783    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.185840 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185818    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185843    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185868    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185896    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6j5\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185923    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185952    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.185979    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.186287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.186009    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.192035 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.192005    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 11:16:24.198467 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.198445    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:16:24.262646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.262558    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"b3d2399fa40543984153a34d3ab95859d12a1246117dc4dcb9579f4dc2b36239"}
Sep 10 11:16:24.262813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.262651    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"48ba6ff5d5ee14790b376c731b1754d65e6c0c4c388215d49da732b06307e39f"}
Sep 10 11:16:24.262813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.262692    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"165eb4e2b5ad1b5ac4d356ea52a882b1f3a2701b750459ac04a6134feb964bf7"}
Sep 10 11:16:24.264703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.264670    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc" event={"ID":"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1","Type":"ContainerStarted","Data":"21d05124f30b4b5c6c646c0a8d44a268c25e80ee1fcd0a497142ebea03bc83c3"}
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287197    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287242    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287278    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287329    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287356    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6j5\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287385    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287413    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287441    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287470    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287499    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287546    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.287851 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.287590    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.288328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288141    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.288328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288266    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.288560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288532    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288756    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288832    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288881    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.288941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.289021    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.289445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.289050    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.290603 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.290092    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.292601 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.290906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.292601 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.291416    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.292601 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.292022    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.292819 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.292731    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.293745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.293663    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.294953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.294577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.294953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.294784    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.294953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.294893    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.294953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.294916    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.295202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.295082    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.295202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.295119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.295733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.295691    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.296147 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.296062    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.296655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.296628    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.298066 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.298025    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.298252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.298233    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6j5\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5\") pod \"prometheus-k8s-0\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.482963 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.482628    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:24.813171 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:24.812859    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:16:24.815688 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:24.815663    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf37b1292_6964_4a39_a599_8cac8010afa5.slice/crio-8b989777f0cbf473ee3dda4736db5d511cf677db9f18929ab8a88deb1830221d WatchSource:0}: Error finding container 8b989777f0cbf473ee3dda4736db5d511cf677db9f18929ab8a88deb1830221d: Status 404 returned error can't find the container with id 8b989777f0cbf473ee3dda4736db5d511cf677db9f18929ab8a88deb1830221d
Sep 10 11:16:25.270987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.270958    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"8b989777f0cbf473ee3dda4736db5d511cf677db9f18929ab8a88deb1830221d"}
Sep 10 11:16:25.274577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.274550    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"468a05bc52748af0b39f26474eeebbae126912995ab46f4a187db7ae30d62a51"}
Sep 10 11:16:25.274669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.274612    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"0dc498a652ea558d3e3068828317c93ebfd6418cc45970334e5fff674010cf15"}
Sep 10 11:16:25.274669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.274640    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" event={"ID":"44447154-eee3-418c-83b0-3e99e0a2f7dd","Type":"ContainerStarted","Data":"b43dadb750daf0002d6a885725ce12289ec3f4ae9fc3b0b4e4c13b1bc8c3585b"}
Sep 10 11:16:25.275354 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.275263    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:25.303811 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:25.303748    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6" podStartSLOduration=2.364407939 podStartE2EDuration="5.303729516s" totalImagesPullingTime="2.939321577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:25.303296377 +0000 UTC m=+120.107363725" watchObservedRunningTime="2026-09-10 11:16:25.303729516 +0000 UTC m=+120.107796863"
Sep 10 11:16:26.281177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.281031    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837" exitCode=0
Sep 10 11:16:26.281177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.281078    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"}
Sep 10 11:16:26.283886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.283852    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc" event={"ID":"d4d70ccf-8fe4-413c-ae67-bbcc066f59b1","Type":"ContainerStarted","Data":"b89c47ef1af634eb8e4d17210369991ea53e705bfd1b903070486a36595fa3d1"}
Sep 10 11:16:26.284200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.284183    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:26.290768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.290740    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc"
Sep 10 11:16:26.337810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:26.337758    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-zrhzc" podStartSLOduration=2.6885136 podStartE2EDuration="4.337742383s" totalImagesPullingTime="1.649228783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:26.335865433 +0000 UTC m=+121.139932781" watchObservedRunningTime="2026-09-10 11:16:26.337742383 +0000 UTC m=+121.141809731"
Sep 10 11:16:31.311743 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311654    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"}
Sep 10 11:16:31.312204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311781    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"}
Sep 10 11:16:31.312204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311825    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"}
Sep 10 11:16:31.312204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311874    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"}
Sep 10 11:16:31.312204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311914    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"}
Sep 10 11:16:31.312204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.311957    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerStarted","Data":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"}
Sep 10 11:16:31.353293 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:31.353229    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.74451544 podStartE2EDuration="7.353206326s" totalImagesPullingTime="5.586585224s" totalInitContainerRuntime="22.105662ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:31.349924348 +0000 UTC m=+126.153991699" watchObservedRunningTime="2026-09-10 11:16:31.353206326 +0000 UTC m=+126.157273673"
Sep 10 11:16:32.296955 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:32.296923    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6ddd57f5d8-6scl6"
Sep 10 11:16:33.979577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:33.979544    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:34.483747 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:34.483713    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:16:34.500031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:34.499991    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:16:34.502627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:34.502601    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/755bead5-09ff-4756-a084-0f327c88a9e1-metrics-certs\") pod \"network-metrics-daemon-wsmgl\" (UID: \"755bead5-09ff-4756-a084-0f327c88a9e1\") " pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:16:34.785266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:34.785181    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9kdff\""
Sep 10 11:16:34.793154 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:34.793120    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wsmgl"
Sep 10 11:16:39.580571 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:39.580487    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wsmgl"]
Sep 10 11:16:39.587047 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:16:39.587006    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755bead5_09ff_4756_a084_0f327c88a9e1.slice/crio-71cabe8cbe2a4f0a5a6651d9b42c13cfb777f3c93e59c723066739f3dee6c182 WatchSource:0}: Error finding container 71cabe8cbe2a4f0a5a6651d9b42c13cfb777f3c93e59c723066739f3dee6c182: Status 404 returned error can't find the container with id 71cabe8cbe2a4f0a5a6651d9b42c13cfb777f3c93e59c723066739f3dee6c182
Sep 10 11:16:40.353546 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:40.353505    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wsmgl" event={"ID":"755bead5-09ff-4756-a084-0f327c88a9e1","Type":"ContainerStarted","Data":"71cabe8cbe2a4f0a5a6651d9b42c13cfb777f3c93e59c723066739f3dee6c182"}
Sep 10 11:16:40.355473 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:40.355428    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-zvmv9" event={"ID":"defa4256-2d6a-44af-ad9b-510d8ae6c24d","Type":"ContainerStarted","Data":"ca462e4c17c1f2d10d95df4f5f7671e7f618e035a931bd0eb782198c47550708"}
Sep 10 11:16:40.355760 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:40.355711    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:40.370047 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:40.370016    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-zvmv9"
Sep 10 11:16:40.381122 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:40.380390    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-zvmv9" podStartSLOduration=1.700025917 podStartE2EDuration="19.380369845s" totalImagesPullingTime="17.680343928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:40.37885109 +0000 UTC m=+135.182918439" watchObservedRunningTime="2026-09-10 11:16:40.380369845 +0000 UTC m=+135.184437194"
Sep 10 11:16:41.363531 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:41.363438    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wsmgl" event={"ID":"755bead5-09ff-4756-a084-0f327c88a9e1","Type":"ContainerStarted","Data":"4821ea3f5a2f65532cce03936d1c118c9364962b7627f463c7d94f33260950ff"}
Sep 10 11:16:41.363531 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:41.363507    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wsmgl" event={"ID":"755bead5-09ff-4756-a084-0f327c88a9e1","Type":"ContainerStarted","Data":"c41c957d9be5d535ccac7e4b8718281192f11422a474837e118abf096fd2478a"}
Sep 10 11:16:41.384293 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:41.384229    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wsmgl" podStartSLOduration=134.143751835 podStartE2EDuration="2m15.384208958s" totalImagesPullingTime="1.240457123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:16:41.381776867 +0000 UTC m=+136.185844216" watchObservedRunningTime="2026-09-10 11:16:41.384208958 +0000 UTC m=+136.188276347"
Sep 10 11:16:47.395517 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:47.395477    2585 generic.go:356] "Generic (PLEG): container finished" podID="7a24adcb-7e33-441f-adac-89488c948515" containerID="7b9340cf1abf065497f528a043875815169a5b1d2231b80910fad20606f49776" exitCode=0
Sep 10 11:16:47.396092 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:47.395560    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9" event={"ID":"7a24adcb-7e33-441f-adac-89488c948515","Type":"ContainerDied","Data":"7b9340cf1abf065497f528a043875815169a5b1d2231b80910fad20606f49776"}
Sep 10 11:16:47.396092 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:47.396009    2585 scope.go:117] "RemoveContainer" containerID="7b9340cf1abf065497f528a043875815169a5b1d2231b80910fad20606f49776"
Sep 10 11:16:48.402379 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:48.402336    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-phkb9" event={"ID":"7a24adcb-7e33-441f-adac-89488c948515","Type":"ContainerStarted","Data":"076938fc716b889ed7e8209ed4365bfa748724ab14ef6488666cb7ca28759ed0"}
Sep 10 11:16:48.998077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:48.998038    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry" containerID="cri-o://b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67" gracePeriod=30
Sep 10 11:16:49.261952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.261881    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:49.333501 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333466    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333514    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333537    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333588    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lwlb\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333618    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333644 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333634    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.333904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333649    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.334113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.333963    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates\") pod \"74fd4771-e64a-4e78-a447-83f8184734ef\" (UID: \"74fd4771-e64a-4e78-a447-83f8184734ef\") "
Sep 10 11:16:49.334113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.334056    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:16:49.334325 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.334253    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-trusted-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.334640 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.334616    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:16:49.336570 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.336535    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb" (OuterVolumeSpecName: "kube-api-access-8lwlb") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "kube-api-access-8lwlb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:16:49.336570 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.336544    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:16:49.336735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.336582    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:16:49.336735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.336622    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:16:49.336735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.336714    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:16:49.343608 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.343579    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "74fd4771-e64a-4e78-a447-83f8184734ef" (UID: "74fd4771-e64a-4e78-a447-83f8184734ef"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:16:49.407110 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.407067    2585 generic.go:356] "Generic (PLEG): container finished" podID="74fd4771-e64a-4e78-a447-83f8184734ef" containerID="b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67" exitCode=0
Sep 10 11:16:49.407110 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.407110    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2"
Sep 10 11:16:49.407661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.407139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" event={"ID":"74fd4771-e64a-4e78-a447-83f8184734ef","Type":"ContainerDied","Data":"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"}
Sep 10 11:16:49.407661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.407187    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6fd9c568-l64w2" event={"ID":"74fd4771-e64a-4e78-a447-83f8184734ef","Type":"ContainerDied","Data":"39b542914ac6f0ae71cb34f80c773f9b9bdbbc1a9205d78d37de47c613ca798d"}
Sep 10 11:16:49.407661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.407210    2585 scope.go:117] "RemoveContainer" containerID="b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"
Sep 10 11:16:49.416134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.416114    2585 scope.go:117] "RemoveContainer" containerID="b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"
Sep 10 11:16:49.416438 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:16:49.416416    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67\": container with ID starting with b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67 not found: ID does not exist" containerID="b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"
Sep 10 11:16:49.416508 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.416444    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67"} err="failed to get container status \"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67\": rpc error: code = NotFound desc = could not find container \"b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67\": container with ID starting with b5003340abeb1899b127c2febc5ad3ca2d63e6c5f29989d8bd32c66a5166df67 not found: ID does not exist"
Sep 10 11:16:49.432165 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.432138    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:16:49.434731 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434712    2585 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-bound-sa-token\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434734    2585 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-registry-tls\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434748    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lwlb\" (UniqueName: \"kubernetes.io/projected/74fd4771-e64a-4e78-a447-83f8184734ef-kube-api-access-8lwlb\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434763    2585 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-image-registry-private-configuration\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434777    2585 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/74fd4771-e64a-4e78-a447-83f8184734ef-ca-trust-extracted\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434789    2585 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/74fd4771-e64a-4e78-a447-83f8184734ef-registry-certificates\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.434826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.434802    2585 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/74fd4771-e64a-4e78-a447-83f8184734ef-installation-pull-secrets\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:16:49.438574 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.438551    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f6fd9c568-l64w2"]
Sep 10 11:16:49.662381 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:49.662345    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" path="/var/lib/kubelet/pods/74fd4771-e64a-4e78-a447-83f8184734ef/volumes"
Sep 10 11:16:58.447904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:58.447870    2585 generic.go:356] "Generic (PLEG): container finished" podID="9b445af6-616b-4a61-aa56-e540042678a4" containerID="9c8562d163198c517634a98ecc85d1bd1341509512b429d1e9323d11153bb0f2" exitCode=0
Sep 10 11:16:58.448328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:58.447922    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn" event={"ID":"9b445af6-616b-4a61-aa56-e540042678a4","Type":"ContainerDied","Data":"9c8562d163198c517634a98ecc85d1bd1341509512b429d1e9323d11153bb0f2"}
Sep 10 11:16:58.448328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:58.448252    2585 scope.go:117] "RemoveContainer" containerID="9c8562d163198c517634a98ecc85d1bd1341509512b429d1e9323d11153bb0f2"
Sep 10 11:16:59.453655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:16:59.453625    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pnftn" event={"ID":"9b445af6-616b-4a61-aa56-e540042678a4","Type":"ContainerStarted","Data":"fc3983b719b635c34fdc951a8ed79bccf26ef88252e49edc9b46102fbbd904be"}
Sep 10 11:17:24.483220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:24.483107    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:24.547506 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:24.547478    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:24.565086 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:24.565062    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:42.702877 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.702840    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703151    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="prometheus" containerID="cri-o://56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54" gracePeriod=600
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703193    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="thanos-sidecar" containerID="cri-o://4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd" gracePeriod=600
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703184    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy" containerID="cri-o://b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56" gracePeriod=600
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703196    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="config-reloader" containerID="cri-o://3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e" gracePeriod=600
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703251    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-web" containerID="cri-o://800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271" gracePeriod=600
Sep 10 11:17:42.703535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.703347    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e" gracePeriod=600
Sep 10 11:17:42.946907 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:42.946883    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.081371 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081280    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081371 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081341    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081371 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081365    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081383    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081399    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081420    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081448    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081474    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081504    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081543    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081574    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081604    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.081637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081633    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081658    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081757    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081819    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081859    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb6j5\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081877    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081904    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081946    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.081977    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.082002    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2\") pod \"f37b1292-6964-4a39-a599-8cac8010afa5\" (UID: \"f37b1292-6964-4a39-a599-8cac8010afa5\") "
Sep 10 11:17:43.082627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.082140    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.082627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.082285    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-metrics-client-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.082627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.082332    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.082627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.082348    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.083263 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.083234    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:17:43.083605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.083434    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.084329 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.083922    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.084329 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.084317    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out" (OuterVolumeSpecName: "config-out") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:17:43.084487 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.084458    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.084871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.084821    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.084976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.084869    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:17:43.085269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.085242    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.085413 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.085391    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.085539 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.085506    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:17:43.085859 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.085829    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.086386 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.086363    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config" (OuterVolumeSpecName: "config") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.086731 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.086692    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.086832 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.086752    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5" (OuterVolumeSpecName: "kube-api-access-kb6j5") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "kube-api-access-kb6j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:17:43.087050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.087029    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.087050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.087042    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.096629 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.096609    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config" (OuterVolumeSpecName: "web-config") pod "f37b1292-6964-4a39-a599-8cac8010afa5" (UID: "f37b1292-6964-4a39-a599-8cac8010afa5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:17:43.183672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183640    2585 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183666    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-web-config\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183677    2585 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-grpc-tls\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183686    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183695    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183703    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183711    2585 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-kube-rbac-proxy\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183720    2585 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-config\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183728    2585 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183739    2585 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-secret-metrics-client-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183747    2585 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f37b1292-6964-4a39-a599-8cac8010afa5-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183755    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-config-out\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183764    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-db\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183771    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183779    2585 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f37b1292-6964-4a39-a599-8cac8010afa5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183787    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-tls-assets\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.183860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.183795    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb6j5\" (UniqueName: \"kubernetes.io/projected/f37b1292-6964-4a39-a599-8cac8010afa5-kube-api-access-kb6j5\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:17:43.625659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625627    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e" exitCode=0
Sep 10 11:17:43.625659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625652    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56" exitCode=0
Sep 10 11:17:43.625659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625658    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271" exitCode=0
Sep 10 11:17:43.625659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625664    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd" exitCode=0
Sep 10 11:17:43.625659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625668    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e" exitCode=0
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625674    2585 generic.go:356] "Generic (PLEG): container finished" podID="f37b1292-6964-4a39-a599-8cac8010afa5" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54" exitCode=0
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625707    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625703    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"}
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625831    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"}
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625861    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"}
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625887    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"}
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625893    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"}
Sep 10 11:17:43.625949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625935    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"}
Sep 10 11:17:43.626333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.625960    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f37b1292-6964-4a39-a599-8cac8010afa5","Type":"ContainerDied","Data":"8b989777f0cbf473ee3dda4736db5d511cf677db9f18929ab8a88deb1830221d"}
Sep 10 11:17:43.635048 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.634717    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.644292 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.644274    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.650764 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.650749    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.657995 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.657978    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.663465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.663438    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:43.667294 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.667278    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.670940 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.670921    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:43.675227 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.675212    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.681691 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.681675    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.681935 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.681916    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.681991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.681944    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.681991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.681966    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.682220 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.682204    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.682260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682224    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.682260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682238    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.682494 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.682479    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.682557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682497    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.682557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682509    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.682747 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.682731    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.682785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682751    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.682785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682764    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.682969 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.682951    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.683004 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682975    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.683004 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.682991    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.683199 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.683184    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.683237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683203    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.683237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683214    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.683449 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.683433    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.683494 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683453    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.683494 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683463    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.683695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683680    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.683695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683694    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.683875 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683860    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.683914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.683874    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.684118 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684099    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.684172 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684118    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.684385 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684363    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.684434 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684387    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.684620 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684604    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.684661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684620    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.684831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684813    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.684871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.684831    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.685043 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685028    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.685085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685043    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.685252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685231    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.685289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685253    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.685472 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685454    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.685472 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685471    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.685679 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685664    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.685720 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685680    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.685886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685872    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.685927 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.685886    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.686081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686065    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.686081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686080    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.686288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686268    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.686347 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686289    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.686551 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686532    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.686605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686551    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.686790 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686772    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.686831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686791    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.687012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.686996    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.687012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687012    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.687228 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687212    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.687264 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687228    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.687462 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687445    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.687462 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687461    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.687674 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687659    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.687715 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687674    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.687866 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687847    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.687910 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.687867    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.688057 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688042    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.688102 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688057    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.688244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688229    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.688285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688244    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.688463 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688447    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.688512 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688462    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.688660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688644    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.688703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688660    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.688846 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688831    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.688887 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.688846    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.689042 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689026    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.689099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689042    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.689262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689246    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.689262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689261    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.689431 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689417    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.689473 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689432    2585 scope.go:117] "RemoveContainer" containerID="fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"
Sep 10 11:17:43.689635 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689616    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e"} err="failed to get container status \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": rpc error: code = NotFound desc = could not find container \"fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e\": container with ID starting with fdff7da174b741c5c1a4089d18adf0bd02839c6d25acbd485e840db5f1f7777e not found: ID does not exist"
Sep 10 11:17:43.689676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689635    2585 scope.go:117] "RemoveContainer" containerID="b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"
Sep 10 11:17:43.689820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689805    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56"} err="failed to get container status \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": rpc error: code = NotFound desc = could not find container \"b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56\": container with ID starting with b973e12ab0271ccec6c37ee4bf244c890e8c168ef51689f5ec1c819349129d56 not found: ID does not exist"
Sep 10 11:17:43.689869 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689820    2585 scope.go:117] "RemoveContainer" containerID="800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"
Sep 10 11:17:43.690008 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.689992    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271"} err="failed to get container status \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": rpc error: code = NotFound desc = could not find container \"800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271\": container with ID starting with 800286bb032d739494c5ba41e99fae50ae2720f7a0cae565d727d01f74f64271 not found: ID does not exist"
Sep 10 11:17:43.690053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690009    2585 scope.go:117] "RemoveContainer" containerID="4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"
Sep 10 11:17:43.690194 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690179    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd"} err="failed to get container status \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": rpc error: code = NotFound desc = could not find container \"4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd\": container with ID starting with 4a7309af72f578400487d91bd0399b8ef023f9bdd9dbc34237ef4073ad0e51dd not found: ID does not exist"
Sep 10 11:17:43.690194 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690193    2585 scope.go:117] "RemoveContainer" containerID="3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"
Sep 10 11:17:43.690409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690390    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e"} err="failed to get container status \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": rpc error: code = NotFound desc = could not find container \"3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e\": container with ID starting with 3cfb1ad53610f552ab9b172190cc2758d9b5817fa14e3ba6cdde80b0caaf130e not found: ID does not exist"
Sep 10 11:17:43.690458 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690410    2585 scope.go:117] "RemoveContainer" containerID="56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"
Sep 10 11:17:43.690589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690573    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54"} err="failed to get container status \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": rpc error: code = NotFound desc = could not find container \"56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54\": container with ID starting with 56f5342fe877d0ce59cf178b4b68a799c5939a0ed7a843c79b9b214765ce1c54 not found: ID does not exist"
Sep 10 11:17:43.690632 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690589    2585 scope.go:117] "RemoveContainer" containerID="55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"
Sep 10 11:17:43.690769 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.690751    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837"} err="failed to get container status \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": rpc error: code = NotFound desc = could not find container \"55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837\": container with ID starting with 55569bf20188bba718b06fb8fd9b589cd1669f53c75e1c1938af0b54bdc13837 not found: ID does not exist"
Sep 10 11:17:43.722191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722170    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:43.722571 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722555    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="prometheus"
Sep 10 11:17:43.722613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722575    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="prometheus"
Sep 10 11:17:43.722613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722592    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="init-config-reloader"
Sep 10 11:17:43.722613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722601    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="init-config-reloader"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722614    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722622    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722633    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-thanos"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722640    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-thanos"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722657    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722665    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722680    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="config-reloader"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722688    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="config-reloader"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722697    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="thanos-sidecar"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722705    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="thanos-sidecar"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722714    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-web"
Sep 10 11:17:43.722721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722721    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-web"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722784    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="config-reloader"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722796    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="prometheus"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722804    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="74fd4771-e64a-4e78-a447-83f8184734ef" containerName="registry"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722814    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="thanos-sidecar"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722822    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-thanos"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722830    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy"
Sep 10 11:17:43.723077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.722841    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" containerName="kube-rbac-proxy-web"
Sep 10 11:17:43.727036 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.727020    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.733634 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733359    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-thanos-sidecar-tls\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" type="*v1.Secret"
Sep 10 11:17:43.733634 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733427    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-rbac-proxy\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" type="*v1.Secret"
Sep 10 11:17:43.733634 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733481    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"prometheus-k8s-0\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="b0cd5e2f-36db-484d-8251-e90ef882a87a" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.733634 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733525    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-tls\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" type="*v1.Secret"
Sep 10 11:17:43.733634 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733568    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-thanos-prometheus-http-client-file\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" type="*v1.Secret"
Sep 10 11:17:43.733923 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733636    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-web-config\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" type="*v1.Secret"
Sep 10 11:17:43.733923 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733679    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-grpc-tls-eovdk7i5quho3\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eovdk7i5quho3\"" type="*v1.Secret"
Sep 10 11:17:43.733923 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733715    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"serving-certs-ca-bundle\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" type="*v1.ConfigMap"
Sep 10 11:17:43.733923 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.733765    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-tls-assets-0\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" type="*v1.Secret"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.734842    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s-dockercfg-ng9t5\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ng9t5\"" type="*v1.Secret"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.734949    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.734949    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"prometheus-trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" type="*v1.ConfigMap"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.734960    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"prometheus-k8s-rulefiles-0\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" type="*v1.ConfigMap"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.735006    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"prometheus-k8s\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" type="*v1.Secret"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:43.735015    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kubelet-serving-ca-bundle\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap"
Sep 10 11:17:43.735043 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.735015    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 11:17:43.735535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.735084    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 11:17:43.735535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.735155    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 11:17:43.763616 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.763591    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:43.788107 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788078    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788111    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788137    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788155    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788192    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788215    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788231    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788256    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788274    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788318    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788336    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788403    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788438    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788460    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhpz\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-kube-api-access-hvhpz\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788475    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788784 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788534    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788784 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788564    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.788784 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.788582    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889454 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889365    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889454 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889419    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889454 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889439    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889454 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889454    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889476    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhpz\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-kube-api-access-hvhpz\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889520    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889638    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889676    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889720    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889743    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.889770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889767    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889789    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889834    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889857    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889879    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889923    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.889943    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.890048    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.890087    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.890203    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.890415    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.890733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.890664    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.891905 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.891884    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0cd5e2f-36db-484d-8251-e90ef882a87a-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.892107 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.892088    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.892348 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.892328    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:43.899165 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:43.899139    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhpz\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-kube-api-access-hvhpz\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.602647 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.602617    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-eovdk7i5quho3\""
Sep 10 11:17:44.612567 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.612535    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.649660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.649635    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 11:17:44.662064 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.662035    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0cd5e2f-36db-484d-8251-e90ef882a87a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.840705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.840671    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 11:17:44.850370 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.850350    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.858254 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.858201    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 11:17:44.862433 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.862414    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.864942 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.864920    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 11:17:44.873412 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.873395    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:44.890439 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890416    2585 configmap.go:193] Couldn't get configMap openshift-monitoring/serving-certs-ca-bundle: failed to sync configmap cache: timed out waiting for the condition
Sep 10 11:17:44.890439 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890431    2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-k8s: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890444    2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-k8s-web-config: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890459    2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-k8s-tls: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890476    2585 secret.go:189] Couldn't get secret openshift-monitoring/kube-rbac-proxy: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890486    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.390465073 +0000 UTC m=+200.194532401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "configmap-serving-certs-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync configmap cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890488    2585 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-k8s-thanos-prometheus-http-client-file: failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890497    2585 configmap.go:193] Couldn't get configMap openshift-monitoring/kubelet-serving-ca-bundle: failed to sync configmap cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890511    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.390493526 +0000 UTC m=+200.194560874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "web-config" (UniqueName: "kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890530    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.390518316 +0000 UTC m=+200.194585654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-kube-rbac-proxy" (UniqueName: "kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890544    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.39053844 +0000 UTC m=+200.194605771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "config" (UniqueName: "kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890554    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.39054896 +0000 UTC m=+200.194616286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "thanos-prometheus-http-client-file" (UniqueName: "kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890563    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.390558847 +0000 UTC m=+200.194626173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-prometheus-k8s-tls" (UniqueName: "kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync secret cache: timed out waiting for the condition
Sep 10 11:17:44.890570 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:17:44.890572    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle podName:b0cd5e2f-36db-484d-8251-e90ef882a87a nodeName:}" failed. No retries permitted until 2026-09-10 11:17:45.390567855 +0000 UTC m=+200.194635181 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "configmap-kubelet-serving-ca-bundle" (UniqueName: "kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle") pod "prometheus-k8s-0" (UID: "b0cd5e2f-36db-484d-8251-e90ef882a87a") : failed to sync configmap cache: timed out waiting for the condition
Sep 10 11:17:44.900766 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.900748    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 11:17:44.943240 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:44.943222    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 11:17:45.044482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.044458    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 11:17:45.053988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.053969    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 11:17:45.221205 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.221183    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ng9t5\""
Sep 10 11:17:45.276277 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.276254    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 11:17:45.296107 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.296087    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 11:17:45.325707 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.325689    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 11:17:45.403101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403076    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403106    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403123    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403165    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403426 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403216    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403426 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403250    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.403903 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403874    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.404012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.403948    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0cd5e2f-36db-484d-8251-e90ef882a87a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.405777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.405753    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.406076 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.406057    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.406116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.406064    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.406148 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.406121    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.406194 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.406145    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0cd5e2f-36db-484d-8251-e90ef882a87a-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0cd5e2f-36db-484d-8251-e90ef882a87a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.540010 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.539934    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:17:45.662045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.662010    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f37b1292-6964-4a39-a599-8cac8010afa5" path="/var/lib/kubelet/pods/f37b1292-6964-4a39-a599-8cac8010afa5/volumes"
Sep 10 11:17:45.841713 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:45.841687    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 11:17:46.639911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:46.639874    2585 generic.go:356] "Generic (PLEG): container finished" podID="b0cd5e2f-36db-484d-8251-e90ef882a87a" containerID="cffde0f9a5f877d595429fce6be3cb26051b46ae97d0a1844a2e9fdc0e61426e" exitCode=0
Sep 10 11:17:46.640059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:46.639957    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerDied","Data":"cffde0f9a5f877d595429fce6be3cb26051b46ae97d0a1844a2e9fdc0e61426e"}
Sep 10 11:17:46.640059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:46.640041    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"d65363340fa4c73a5312f5641ab3c9f8f75b245cdb10862130b84258b8b177f9"}
Sep 10 11:17:47.647590 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647552    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"2cb9fe3da2e6c3952bbdcba0ec45101e8ed3effa2337613911ee61b6060d4813"}
Sep 10 11:17:47.647969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647622    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"c739b4814584f89fd65c663de806461d45739c19f1f43fc81aec418596bec7c6"}
Sep 10 11:17:47.647969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647649    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"f8fd1a826c9fca9202ff6be38304cec73b439051622e93679a8e26e842943995"}
Sep 10 11:17:47.647969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647673    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"1aac78005e0214f6875016e1bdad4a73514ef203cb8c4a28bce42b4b1df7d0a8"}
Sep 10 11:17:47.647969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647702    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"1ad308c92a1ae4d286839f818a2bee5e423f32b948e60b4e3edb8b048116cb9f"}
Sep 10 11:17:47.647969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.647723    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0cd5e2f-36db-484d-8251-e90ef882a87a","Type":"ContainerStarted","Data":"0517b114be9a9cafe3ebe825c91ca848ad5bebb605c30e9b9b46741e5cc9d0c0"}
Sep 10 11:17:47.685957 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:47.685906    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.669565979 podStartE2EDuration="4.685891532s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.325553ms" isStatefulPod=false podCreationTimestamp="2026-09-10 11:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:17:47.67947916 +0000 UTC m=+202.483546507" watchObservedRunningTime="2026-09-10 11:17:47.685891532 +0000 UTC m=+202.489958879"
Sep 10 11:17:50.540347 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:17:50.540282    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:18:45.541075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:18:45.541040    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:18:45.556461 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:18:45.556436    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:18:45.882242 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:18:45.882168    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 11:19:09.708693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.708657    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"]
Sep 10 11:19:09.710712 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.710696    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.712915 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.712889    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 11:19:09.713469 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.713446    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 11:19:09.713739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.713723    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qdhsw\""
Sep 10 11:19:09.725094 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.725071    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"]
Sep 10 11:19:09.843568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.843536    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.843744 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.843612    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7swq\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-kube-api-access-s7swq\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.944507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.944473    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.944639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.944575    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7swq\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-kube-api-access-s7swq\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.952848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.952824    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:09.953204 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:09.953184    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7swq\" (UniqueName: \"kubernetes.io/projected/02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4-kube-api-access-s7swq\") pod \"cert-manager-cainjector-559cdd8c87-jmv9b\" (UID: \"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:10.039711 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:10.039634    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"
Sep 10 11:19:10.309590 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:10.309510    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b"]
Sep 10 11:19:10.312638 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:19:10.312607    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02c8f0bf_9ae6_45cf_8a02_033c4a7f6de4.slice/crio-a9e905178645a9ce88d09ffb279b413831da180894419bc5b89352c93c679b50 WatchSource:0}: Error finding container a9e905178645a9ce88d09ffb279b413831da180894419bc5b89352c93c679b50: Status 404 returned error can't find the container with id a9e905178645a9ce88d09ffb279b413831da180894419bc5b89352c93c679b50
Sep 10 11:19:10.959537 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:10.959504    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b" event={"ID":"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4","Type":"ContainerStarted","Data":"a9e905178645a9ce88d09ffb279b413831da180894419bc5b89352c93c679b50"}
Sep 10 11:19:13.974073 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:13.974033    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b" event={"ID":"02c8f0bf-9ae6-45cf-8a02-033c4a7f6de4","Type":"ContainerStarted","Data":"b909c2ea0e55c618ca0a414364341ae7bd7889b08c965847448dd44c6444d8f7"}
Sep 10 11:19:13.990189 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:13.990115    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jmv9b" podStartSLOduration=1.981097005 podStartE2EDuration="4.99009684s" totalImagesPullingTime="3.008999835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:19:13.989778563 +0000 UTC m=+288.793845910" watchObservedRunningTime="2026-09-10 11:19:13.99009684 +0000 UTC m=+288.794164188"
Sep 10 11:19:25.575785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:25.575757    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:19:25.580127 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:25.580106    2585 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 11:19:25.580269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:25.580250    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:19:43.859920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.859885    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"]
Sep 10 11:19:43.862959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.862935    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:43.865513 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.865492    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 11:19:43.865631 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.865496    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-789bb\""
Sep 10 11:19:43.865804 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.865788    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 11:19:43.865880 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.865806    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 11:19:43.865880 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.865813    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 11:19:43.866085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.866065    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 11:19:43.893141 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.893120    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"]
Sep 10 11:19:43.908987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.908965    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-metrics-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:43.909092 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.909018    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rlsm\" (UniqueName: \"kubernetes.io/projected/a0b2f34c-43c1-4902-a101-05df37397122-kube-api-access-9rlsm\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:43.909092 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.909048    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0b2f34c-43c1-4902-a101-05df37397122-manager-config\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:43.909092 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:43.909084    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.010065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.010035    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0b2f34c-43c1-4902-a101-05df37397122-manager-config\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.010242 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.010071    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.010242 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.010100    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-metrics-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.010242 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.010239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rlsm\" (UniqueName: \"kubernetes.io/projected/a0b2f34c-43c1-4902-a101-05df37397122-kube-api-access-9rlsm\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.010746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.010727    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0b2f34c-43c1-4902-a101-05df37397122-manager-config\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.012604 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.012582    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.012693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.012639    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0b2f34c-43c1-4902-a101-05df37397122-metrics-cert\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.021042 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.021013    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rlsm\" (UniqueName: \"kubernetes.io/projected/a0b2f34c-43c1-4902-a101-05df37397122-kube-api-access-9rlsm\") pod \"lws-controller-manager-7b89dcdb5f-4v9ls\" (UID: \"a0b2f34c-43c1-4902-a101-05df37397122\") " pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.172763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.172737    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:44.440941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.440875    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"]
Sep 10 11:19:44.443768 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:19:44.443740    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b2f34c_43c1_4902_a101_05df37397122.slice/crio-fba919f123e6bff256cdacc39b179f6d993175d8434ce5e6f67410a36861c6da WatchSource:0}: Error finding container fba919f123e6bff256cdacc39b179f6d993175d8434ce5e6f67410a36861c6da: Status 404 returned error can't find the container with id fba919f123e6bff256cdacc39b179f6d993175d8434ce5e6f67410a36861c6da
Sep 10 11:19:44.445698 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:44.445680    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:19:45.094747 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:45.094712    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls" event={"ID":"a0b2f34c-43c1-4902-a101-05df37397122","Type":"ContainerStarted","Data":"fba919f123e6bff256cdacc39b179f6d993175d8434ce5e6f67410a36861c6da"}
Sep 10 11:19:48.107369 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:48.107326    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls" event={"ID":"a0b2f34c-43c1-4902-a101-05df37397122","Type":"ContainerStarted","Data":"ce16a4284b7abd43d37a1b390f22a0807180e0d3e1f9d5d01da86061ddd926b4"}
Sep 10 11:19:48.107757 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:48.107435    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:19:48.128639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:48.128586    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls" podStartSLOduration=2.28739734 podStartE2EDuration="5.128571915s" totalImagesPullingTime="2.841174575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:19:48.127967304 +0000 UTC m=+322.932034674" watchObservedRunningTime="2026-09-10 11:19:48.128571915 +0000 UTC m=+322.932639262"
Sep 10 11:19:56.354956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.354927    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"]
Sep 10 11:19:56.358210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.358193    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.361193 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.361168    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zxnsf\""
Sep 10 11:19:56.361336 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.361248    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 11:19:56.361405 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.361337    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 11:19:56.361405 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.361342    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 11:19:56.361625 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.361605    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 11:19:56.383873 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.383850    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"]
Sep 10 11:19:56.414217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.414195    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwc4\" (UniqueName: \"kubernetes.io/projected/6e2b0daa-2309-427c-aab7-ebc194807f00-kube-api-access-rhwc4\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.414372 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.414256    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-tmp\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.414372 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.414351    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-manifests\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.414477 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.414386    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.414477 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.414418    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-webhook-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515503    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-tmp\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515538    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-manifests\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515561    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515580    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-webhook-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515617    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwc4\" (UniqueName: \"kubernetes.io/projected/6e2b0daa-2309-427c-aab7-ebc194807f00-kube-api-access-rhwc4\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.515939 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.515914    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-manifests\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.517836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.517815    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e2b0daa-2309-427c-aab7-ebc194807f00-tmp\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.518149 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.518130    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.518201 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.518136    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6e2b0daa-2309-427c-aab7-ebc194807f00-webhook-cert\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.523769 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.523750    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwc4\" (UniqueName: \"kubernetes.io/projected/6e2b0daa-2309-427c-aab7-ebc194807f00-kube-api-access-rhwc4\") pod \"opendatahub-operator-controller-manager-d6d6998d4-d2479\" (UID: \"6e2b0daa-2309-427c-aab7-ebc194807f00\") " pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.668168 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.668145    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:19:56.957326 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:56.957230    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"]
Sep 10 11:19:56.960520 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:19:56.960492    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2b0daa_2309_427c_aab7_ebc194807f00.slice/crio-4a711d7ddbd3c12191abecd56f87bf4fb7b3bf0ee35a9a1517638b2fa528fb01 WatchSource:0}: Error finding container 4a711d7ddbd3c12191abecd56f87bf4fb7b3bf0ee35a9a1517638b2fa528fb01: Status 404 returned error can't find the container with id 4a711d7ddbd3c12191abecd56f87bf4fb7b3bf0ee35a9a1517638b2fa528fb01
Sep 10 11:19:57.143261 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:57.143227    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479" event={"ID":"6e2b0daa-2309-427c-aab7-ebc194807f00","Type":"ContainerStarted","Data":"4a711d7ddbd3c12191abecd56f87bf4fb7b3bf0ee35a9a1517638b2fa528fb01"}
Sep 10 11:19:59.113950 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:19:59.113914    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b89dcdb5f-4v9ls"
Sep 10 11:20:01.169260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:01.169226    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479" event={"ID":"6e2b0daa-2309-427c-aab7-ebc194807f00","Type":"ContainerStarted","Data":"57aafdeacfca45a02815151e8d5eca1f533ca4fd5d8d58e29a55dd73cc31596c"}
Sep 10 11:20:02.174741 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:02.174709    2585 generic.go:356] "Generic (PLEG): container finished" podID="6e2b0daa-2309-427c-aab7-ebc194807f00" containerID="57aafdeacfca45a02815151e8d5eca1f533ca4fd5d8d58e29a55dd73cc31596c" exitCode=0
Sep 10 11:20:02.175104 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:02.174761    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479" event={"ID":"6e2b0daa-2309-427c-aab7-ebc194807f00","Type":"ContainerDied","Data":"57aafdeacfca45a02815151e8d5eca1f533ca4fd5d8d58e29a55dd73cc31596c"}
Sep 10 11:20:03.180614 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:03.180578    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479" event={"ID":"6e2b0daa-2309-427c-aab7-ebc194807f00","Type":"ContainerStarted","Data":"aaf778c1dc059dd6e03b08bc707cccd421dd6e37b7296c520a94865afc663d5e"}
Sep 10 11:20:03.181010 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:03.180688    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:20:03.208215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:03.208158    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479" podStartSLOduration=2.382755358 podStartE2EDuration="7.208140724s" totalImagesPullingTime="3.821281581s" totalInitContainerRuntime="1.004103785s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:19:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:20:03.206524559 +0000 UTC m=+338.010591906" watchObservedRunningTime="2026-09-10 11:20:03.208140724 +0000 UTC m=+338.012208071"
Sep 10 11:20:14.186961 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:14.186926    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d6d6998d4-d2479"
Sep 10 11:20:27.528561 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.528486    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"]
Sep 10 11:20:27.533422 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.533401    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.535351 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.535331    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 11:20:27.536625 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.536597    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 11:20:27.536735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.536713    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 11:20:27.537119 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.537097    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f2t8t\""
Sep 10 11:20:27.537207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.537100    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 11:20:27.550778 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.550756    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"]
Sep 10 11:20:27.569629 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.569608    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6qk2\" (UniqueName: \"kubernetes.io/projected/2319856e-6de0-4223-8c4c-5e76f83a83df-kube-api-access-m6qk2\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.569738 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.569684    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2319856e-6de0-4223-8c4c-5e76f83a83df-tmp\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.569738 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.569710    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.671000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.670952    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6qk2\" (UniqueName: \"kubernetes.io/projected/2319856e-6de0-4223-8c4c-5e76f83a83df-kube-api-access-m6qk2\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.671202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.671059    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2319856e-6de0-4223-8c4c-5e76f83a83df-tmp\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.671202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.671096    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.671356 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:20:27.671214    2585 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 11:20:27.671356 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:20:27.671277    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs podName:2319856e-6de0-4223-8c4c-5e76f83a83df nodeName:}" failed. No retries permitted until 2026-09-10 11:20:28.171257715 +0000 UTC m=+362.975325057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs") pod "kube-auth-proxy-5f4449d5c7-r8559" (UID: "2319856e-6de0-4223-8c4c-5e76f83a83df") : secret "kube-auth-proxy-tls" not found
Sep 10 11:20:27.673262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.673232    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2319856e-6de0-4223-8c4c-5e76f83a83df-tmp\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:27.684448 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:27.684426    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6qk2\" (UniqueName: \"kubernetes.io/projected/2319856e-6de0-4223-8c4c-5e76f83a83df-kube-api-access-m6qk2\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:28.174977 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:28.174939    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:28.177425 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:28.177396    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2319856e-6de0-4223-8c4c-5e76f83a83df-tls-certs\") pod \"kube-auth-proxy-5f4449d5c7-r8559\" (UID: \"2319856e-6de0-4223-8c4c-5e76f83a83df\") " pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:28.443137 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:28.443067    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"
Sep 10 11:20:28.723856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:28.723778    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559"]
Sep 10 11:20:28.728059 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:20:28.728027    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2319856e_6de0_4223_8c4c_5e76f83a83df.slice/crio-ea484014abfa3dddcb01282f69ede43d6b7fc3ecaece1611daec69de54ff8e38 WatchSource:0}: Error finding container ea484014abfa3dddcb01282f69ede43d6b7fc3ecaece1611daec69de54ff8e38: Status 404 returned error can't find the container with id ea484014abfa3dddcb01282f69ede43d6b7fc3ecaece1611daec69de54ff8e38
Sep 10 11:20:29.292892 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:29.292847    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559" event={"ID":"2319856e-6de0-4223-8c4c-5e76f83a83df","Type":"ContainerStarted","Data":"ea484014abfa3dddcb01282f69ede43d6b7fc3ecaece1611daec69de54ff8e38"}
Sep 10 11:20:32.308585 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:32.308551    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559" event={"ID":"2319856e-6de0-4223-8c4c-5e76f83a83df","Type":"ContainerStarted","Data":"64439bb4da71697728d5965da0238fa2207466586f67e04fb6567767717c2fdf"}
Sep 10 11:20:32.338560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:20:32.338513    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f4449d5c7-r8559" podStartSLOduration=2.794803066 podStartE2EDuration="5.338496985s" totalImagesPullingTime="2.543693919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:20:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:20:32.337910976 +0000 UTC m=+367.141978325" watchObservedRunningTime="2026-09-10 11:20:32.338496985 +0000 UTC m=+367.142564332"
Sep 10 11:21:54.186885 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.186813    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"]
Sep 10 11:21:54.190118 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.190102    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:54.192716 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.192694    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-flc94\""
Sep 10 11:21:54.192821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.192728    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 11:21:54.192821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.192728    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 11:21:54.193087 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.193069    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 11:21:54.202785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.202761    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"]
Sep 10 11:21:54.298976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.298940    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncfcn\" (UniqueName: \"kubernetes.io/projected/0c9e4297-1c9e-4cdb-b16f-dd19348fcf41-kube-api-access-ncfcn\") pod \"dns-operator-controller-manager-7b84cd7c9c-652w5\" (UID: \"0c9e4297-1c9e-4cdb-b16f-dd19348fcf41\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:54.399955 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.399927    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncfcn\" (UniqueName: \"kubernetes.io/projected/0c9e4297-1c9e-4cdb-b16f-dd19348fcf41-kube-api-access-ncfcn\") pod \"dns-operator-controller-manager-7b84cd7c9c-652w5\" (UID: \"0c9e4297-1c9e-4cdb-b16f-dd19348fcf41\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:54.411313 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.411270    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncfcn\" (UniqueName: \"kubernetes.io/projected/0c9e4297-1c9e-4cdb-b16f-dd19348fcf41-kube-api-access-ncfcn\") pod \"dns-operator-controller-manager-7b84cd7c9c-652w5\" (UID: \"0c9e4297-1c9e-4cdb-b16f-dd19348fcf41\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:54.500744 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.500673    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:54.777574 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:54.777542    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"]
Sep 10 11:21:54.781347 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:21:54.781319    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9e4297_1c9e_4cdb_b16f_dd19348fcf41.slice/crio-e71daf287344c632c2414a6267c2ec91afe0ce5bdf4d7252e468573706062ecc WatchSource:0}: Error finding container e71daf287344c632c2414a6267c2ec91afe0ce5bdf4d7252e468573706062ecc: Status 404 returned error can't find the container with id e71daf287344c632c2414a6267c2ec91afe0ce5bdf4d7252e468573706062ecc
Sep 10 11:21:55.666340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:55.666286    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5" event={"ID":"0c9e4297-1c9e-4cdb-b16f-dd19348fcf41","Type":"ContainerStarted","Data":"e71daf287344c632c2414a6267c2ec91afe0ce5bdf4d7252e468573706062ecc"}
Sep 10 11:21:56.400469 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.400433    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"]
Sep 10 11:21:56.405195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.405177    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:56.409032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.409010    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-96jkp\""
Sep 10 11:21:56.423167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.423140    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"]
Sep 10 11:21:56.517940 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.517905    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2dxv\" (UniqueName: \"kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv\") pod \"limitador-operator-controller-manager-74495c69f8-9t9kd\" (UID: \"bdeb1732-e0ba-4368-9f21-1188c41d52f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:56.619034 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.619005    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2dxv\" (UniqueName: \"kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv\") pod \"limitador-operator-controller-manager-74495c69f8-9t9kd\" (UID: \"bdeb1732-e0ba-4368-9f21-1188c41d52f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:56.633563 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.633504    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2dxv\" (UniqueName: \"kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv\") pod \"limitador-operator-controller-manager-74495c69f8-9t9kd\" (UID: \"bdeb1732-e0ba-4368-9f21-1188c41d52f2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:56.717836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:56.717810    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:57.465963 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:57.465884    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"]
Sep 10 11:21:57.468885 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:21:57.468858    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdeb1732_e0ba_4368_9f21_1188c41d52f2.slice/crio-4d0b7ef89e98f48b47ba7725a854af6277bd616d35bfa5aceb1a4ad4cffb4bae WatchSource:0}: Error finding container 4d0b7ef89e98f48b47ba7725a854af6277bd616d35bfa5aceb1a4ad4cffb4bae: Status 404 returned error can't find the container with id 4d0b7ef89e98f48b47ba7725a854af6277bd616d35bfa5aceb1a4ad4cffb4bae
Sep 10 11:21:57.677264 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:57.677234    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5" event={"ID":"0c9e4297-1c9e-4cdb-b16f-dd19348fcf41","Type":"ContainerStarted","Data":"b59b6f7646fc4d72b4d0fe646b7ad1f1964cf67daca0192d1f6852bf5ab1157e"}
Sep 10 11:21:57.677463 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:57.677359    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:21:57.678587 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:57.678561    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd" event={"ID":"bdeb1732-e0ba-4368-9f21-1188c41d52f2","Type":"ContainerStarted","Data":"4d0b7ef89e98f48b47ba7725a854af6277bd616d35bfa5aceb1a4ad4cffb4bae"}
Sep 10 11:21:57.709127 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:57.709081    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5" podStartSLOduration=1.222417358 podStartE2EDuration="3.709065908s" totalImagesPullingTime="2.48664855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:21:57.708117695 +0000 UTC m=+452.512185044" watchObservedRunningTime="2026-09-10 11:21:57.709065908 +0000 UTC m=+452.513133255"
Sep 10 11:21:59.689374 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:59.689336    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd" event={"ID":"bdeb1732-e0ba-4368-9f21-1188c41d52f2","Type":"ContainerStarted","Data":"fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535"}
Sep 10 11:21:59.689833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:59.689494    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:21:59.721105 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:21:59.721055    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd" podStartSLOduration=1.8014765719999999 podStartE2EDuration="3.721040847s" totalImagesPullingTime="1.919564275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:21:59.719535728 +0000 UTC m=+454.523603077" watchObservedRunningTime="2026-09-10 11:21:59.721040847 +0000 UTC m=+454.525108196"
Sep 10 11:22:08.685613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:08.685581    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-652w5"
Sep 10 11:22:10.120288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.120256    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"]
Sep 10 11:22:10.120706 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.120456    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" containerName="manager" containerID="cri-o://fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535" gracePeriod=2
Sep 10 11:22:10.122072 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.122049    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.133748 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.133726    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"]
Sep 10 11:22:10.146279 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.146249    2585 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bdeb1732-e0ba-4368-9f21-1188c41d52f2\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-74495c69f8-9t9kd\": pods \"limitador-operator-controller-manager-74495c69f8-9t9kd\" not found" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.149057 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.149030    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9t9kd\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.361824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.361799    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.365440 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.365414    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9t9kd\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.441495 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.441464    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2dxv\" (UniqueName: \"kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv\") pod \"bdeb1732-e0ba-4368-9f21-1188c41d52f2\" (UID: \"bdeb1732-e0ba-4368-9f21-1188c41d52f2\") "
Sep 10 11:22:10.443588 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.443557    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv" (OuterVolumeSpecName: "kube-api-access-n2dxv") pod "bdeb1732-e0ba-4368-9f21-1188c41d52f2" (UID: "bdeb1732-e0ba-4368-9f21-1188c41d52f2"). InnerVolumeSpecName "kube-api-access-n2dxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:22:10.542828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.542804    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2dxv\" (UniqueName: \"kubernetes.io/projected/bdeb1732-e0ba-4368-9f21-1188c41d52f2-kube-api-access-n2dxv\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:22:10.737717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.737637    2585 generic.go:356] "Generic (PLEG): container finished" podID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" containerID="fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535" exitCode=0
Sep 10 11:22:10.737717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.737683    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.737922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.737736    2585 scope.go:117] "RemoveContainer" containerID="fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535"
Sep 10 11:22:10.740611 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.740582    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9t9kd\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:10.746786 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.746769    2585 scope.go:117] "RemoveContainer" containerID="fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535"
Sep 10 11:22:10.747054 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.747038    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535\": container with ID starting with fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535 not found: ID does not exist" containerID="fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535"
Sep 10 11:22:10.747156 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:10.747063    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535"} err="failed to get container status \"fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535\": rpc error: code = NotFound desc = could not find container \"fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535\": container with ID starting with fd15d819f18739e6a04af56c0fafa13add2b4873e9a660b06c7e3c4915cf3535 not found: ID does not exist"
Sep 10 11:22:10.753443 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:22:10.753421    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-9t9kd\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9t9kd"
Sep 10 11:22:11.662402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:11.662367    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" path="/var/lib/kubelet/pods/bdeb1732-e0ba-4368-9f21-1188c41d52f2/volumes"
Sep 10 11:22:12.216534 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.216498    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:22:12.216834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.216822    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" containerName="manager"
Sep 10 11:22:12.216886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.216835    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" containerName="manager"
Sep 10 11:22:12.216928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.216892    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdeb1732-e0ba-4368-9f21-1188c41d52f2" containerName="manager"
Sep 10 11:22:12.221108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.221089    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.226743 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.226716    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ngvwc\""
Sep 10 11:22:12.248624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.248590    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:22:12.257003 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.256974    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.257143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.257090    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skr6m\" (UniqueName: \"kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.358079 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.358042    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skr6m\" (UniqueName: \"kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.358252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.358101    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.358456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.358435    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.369097 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.369064    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skr6m\" (UniqueName: \"kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m\") pod \"kuadrant-operator-controller-manager-5769f99d9f-k7x72\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.531044 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.530969    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:12.814244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:12.814216    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:22:12.816711 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:22:12.816687    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c32ebb_5818_4132_bac4_f9e59f6376bc.slice/crio-ccb5ccadab7cd4352e738d354572323d92ab1b03994b7fed56b73b7c5756c205 WatchSource:0}: Error finding container ccb5ccadab7cd4352e738d354572323d92ab1b03994b7fed56b73b7c5756c205: Status 404 returned error can't find the container with id ccb5ccadab7cd4352e738d354572323d92ab1b03994b7fed56b73b7c5756c205
Sep 10 11:22:13.753924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:13.753885    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" event={"ID":"11c32ebb-5818-4132-bac4-f9e59f6376bc","Type":"ContainerStarted","Data":"ccb5ccadab7cd4352e738d354572323d92ab1b03994b7fed56b73b7c5756c205"}
Sep 10 11:22:16.770178 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:16.770139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" event={"ID":"11c32ebb-5818-4132-bac4-f9e59f6376bc","Type":"ContainerStarted","Data":"cb2b5bb61315638bd7214b73aae8784f174249212d24da00c684aa2d54e3c986"}
Sep 10 11:22:16.770558 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:16.770246    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:16.801815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:16.801768    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" podStartSLOduration=1.3087939880000001 podStartE2EDuration="4.801752063s" totalImagesPullingTime="3.492958075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:22:16.799550152 +0000 UTC m=+471.603617513" watchObservedRunningTime="2026-09-10 11:22:16.801752063 +0000 UTC m=+471.605819413"
Sep 10 11:22:27.776948 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:27.776918    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:22:47.217543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.217505    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:22:47.268340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.268289    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:22:47.268500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.268436    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.270188 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.270165    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:22:47.274562 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.274543    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 11:22:47.274685 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.274663    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-75ljd\""
Sep 10 11:22:47.351461 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.351436    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/346e98ca-5281-468a-8dc9-5373094d614f-config-file\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.351586 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.351482    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbwn8\" (UniqueName: \"kubernetes.io/projected/346e98ca-5281-468a-8dc9-5373094d614f-kube-api-access-wbwn8\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.452649 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.452617    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbwn8\" (UniqueName: \"kubernetes.io/projected/346e98ca-5281-468a-8dc9-5373094d614f-kube-api-access-wbwn8\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.452788 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.452775    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/346e98ca-5281-468a-8dc9-5373094d614f-config-file\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.453345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.453297    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/346e98ca-5281-468a-8dc9-5373094d614f-config-file\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.461031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.461010    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbwn8\" (UniqueName: \"kubernetes.io/projected/346e98ca-5281-468a-8dc9-5373094d614f-kube-api-access-wbwn8\") pod \"limitador-limitador-84994f7ffc-7d7xq\" (UID: \"346e98ca-5281-468a-8dc9-5373094d614f\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.579032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.578956    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:47.872112 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.872038    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:22:47.875403 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:22:47.875371    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346e98ca_5281_468a_8dc9_5373094d614f.slice/crio-d6adc264009ca28a1308c88dd9311a8df0a410c11ed5eeea753d864e2be5a586 WatchSource:0}: Error finding container d6adc264009ca28a1308c88dd9311a8df0a410c11ed5eeea753d864e2be5a586: Status 404 returned error can't find the container with id d6adc264009ca28a1308c88dd9311a8df0a410c11ed5eeea753d864e2be5a586
Sep 10 11:22:47.908526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:47.908495    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq" event={"ID":"346e98ca-5281-468a-8dc9-5373094d614f","Type":"ContainerStarted","Data":"d6adc264009ca28a1308c88dd9311a8df0a410c11ed5eeea753d864e2be5a586"}
Sep 10 11:22:49.920356 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:49.920320    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq" event={"ID":"346e98ca-5281-468a-8dc9-5373094d614f","Type":"ContainerStarted","Data":"77d4c671d368db92c14fb77e81bea9d7d89a15583c6945170257176edb477b48"}
Sep 10 11:22:49.920733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:49.920415    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:22:49.939842 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:22:49.939789    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq" podStartSLOduration=1.814100217 podStartE2EDuration="2.939775262s" totalImagesPullingTime="1.125675045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:22:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:22:49.939512343 +0000 UTC m=+504.743579690" watchObservedRunningTime="2026-09-10 11:22:49.939775262 +0000 UTC m=+504.743842650"
Sep 10 11:23:00.926768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:23:00.926739    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"
Sep 10 11:24:00.889225 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:00.889190    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:24:17.768896 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:17.768862    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:24:25.609450 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:25.609421    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:24:25.610717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:25.610693    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:24:31.961289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:31.961251    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:24:46.257144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:46.257104    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:24:56.456325 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:24:56.456227    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:25:06.060215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:06.060174    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:25:14.046442 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.046400    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"]
Sep 10 11:25:14.049954 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.049932    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.052702 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.052676    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 11:25:14.052843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.052674    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 11:25:14.052843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.052680    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 11:25:14.052843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.052685    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5ft7p\""
Sep 10 11:25:14.060515 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.060490    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"]
Sep 10 11:25:14.189453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189408    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.189656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189469    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zqcq\" (UniqueName: \"kubernetes.io/projected/a0218e53-3223-4948-932c-f03d9f27e617-kube-api-access-6zqcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.189656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189511    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0218e53-3223-4948-932c-f03d9f27e617-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.189656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189529    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.189656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189609    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.189656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.189644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290386 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290351    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290422    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290472    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zqcq\" (UniqueName: \"kubernetes.io/projected/a0218e53-3223-4948-932c-f03d9f27e617-kube-api-access-6zqcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290512    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0218e53-3223-4948-932c-f03d9f27e617-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290578    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290808    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290842    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.290954 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.290894    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.292792 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.292770    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0218e53-3223-4948-932c-f03d9f27e617-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.293052 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.293031    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0218e53-3223-4948-932c-f03d9f27e617-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.298184 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.298119    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zqcq\" (UniqueName: \"kubernetes.io/projected/a0218e53-3223-4948-932c-f03d9f27e617-kube-api-access-6zqcq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9mdrl\" (UID: \"a0218e53-3223-4948-932c-f03d9f27e617\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.362524 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.362486    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:14.636834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.636803    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"]
Sep 10 11:25:14.639532 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:25:14.639508    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0218e53_3223_4948_932c_f03d9f27e617.slice/crio-d3b4e10b11694288b23a6914e122ae577a0864fca5697df4b40279a23c49f8b7 WatchSource:0}: Error finding container d3b4e10b11694288b23a6914e122ae577a0864fca5697df4b40279a23c49f8b7: Status 404 returned error can't find the container with id d3b4e10b11694288b23a6914e122ae577a0864fca5697df4b40279a23c49f8b7
Sep 10 11:25:14.640894 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:14.640877    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:25:15.573904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:15.573864    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl" event={"ID":"a0218e53-3223-4948-932c-f03d9f27e617","Type":"ContainerStarted","Data":"d3b4e10b11694288b23a6914e122ae577a0864fca5697df4b40279a23c49f8b7"}
Sep 10 11:25:16.165764 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.165729    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:25:16.236730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.236702    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"]
Sep 10 11:25:16.240154 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.240133    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.242002 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.241984    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 11:25:16.248850 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.248818    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"]
Sep 10 11:25:16.307482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307456    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.307482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307484    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.307651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307532    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz79v\" (UniqueName: \"kubernetes.io/projected/55d2e410-6bb8-4327-8fac-237a3593d850-kube-api-access-sz79v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.307651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307558    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.307651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307590    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d2e410-6bb8-4327-8fac-237a3593d850-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.307651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.307617    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408515 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408485    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408583    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408614    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz79v\" (UniqueName: \"kubernetes.io/projected/55d2e410-6bb8-4327-8fac-237a3593d850-kube-api-access-sz79v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408661    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d2e410-6bb8-4327-8fac-237a3593d850-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.408911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408883    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.409020 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.408999    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.409091 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.409050    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.410779 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.410755    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55d2e410-6bb8-4327-8fac-237a3593d850-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.411007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.410990    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55d2e410-6bb8-4327-8fac-237a3593d850-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.417155 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.417098    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz79v\" (UniqueName: \"kubernetes.io/projected/55d2e410-6bb8-4327-8fac-237a3593d850-kube-api-access-sz79v\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2\" (UID: \"55d2e410-6bb8-4327-8fac-237a3593d850\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.549622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.549585    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:16.580868 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.580834    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl" event={"ID":"a0218e53-3223-4948-932c-f03d9f27e617","Type":"ContainerStarted","Data":"2d07e7e9c69790d25f124b327eee3d7a027e33ba220b2d1117129d3939116fa6"}
Sep 10 11:25:16.600217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.599608    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl" podStartSLOduration=1.439671255 podStartE2EDuration="2.599587129s" totalImagesPullingTime="1.159915874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:25:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:25:16.597215109 +0000 UTC m=+651.401282489" watchObservedRunningTime="2026-09-10 11:25:16.599587129 +0000 UTC m=+651.403654478"
Sep 10 11:25:16.824548 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:16.824152    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"]
Sep 10 11:25:16.827088 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:25:16.827057    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55d2e410_6bb8_4327_8fac_237a3593d850.slice/crio-415437f4c45c2ab8f9ec6efb63056f64096f7e5daec320cf2637a185287ec98e WatchSource:0}: Error finding container 415437f4c45c2ab8f9ec6efb63056f64096f7e5daec320cf2637a185287ec98e: Status 404 returned error can't find the container with id 415437f4c45c2ab8f9ec6efb63056f64096f7e5daec320cf2637a185287ec98e
Sep 10 11:25:17.586871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:17.586836    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2" event={"ID":"55d2e410-6bb8-4327-8fac-237a3593d850","Type":"ContainerStarted","Data":"989027e276591919e3cfd19ba84ae9ce0301a10aca9a3b19062191ca896bf675"}
Sep 10 11:25:17.587219 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:17.586881    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2" event={"ID":"55d2e410-6bb8-4327-8fac-237a3593d850","Type":"ContainerStarted","Data":"415437f4c45c2ab8f9ec6efb63056f64096f7e5daec320cf2637a185287ec98e"}
Sep 10 11:25:17.606272 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:17.606223    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2" podStartSLOduration=1.4185345950000001 podStartE2EDuration="1.606207079s" totalImagesPullingTime="187.672484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:25:17.604875671 +0000 UTC m=+652.408943019" watchObservedRunningTime="2026-09-10 11:25:17.606207079 +0000 UTC m=+652.410274427"
Sep 10 11:25:23.357713 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:23.357680    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:25:24.363192 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:24.363159    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:24.363607 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:24.363464    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:24.375508 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:24.375479    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:24.631057 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:24.630975    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9mdrl"
Sep 10 11:25:26.550571 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:26.550540    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:26.550972 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:26.550589    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:26.562930 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:26.562906    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:25:26.639377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:25:26.639346    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2"
Sep 10 11:26:49.216832 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.216748    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:49.220128 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.220109    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:49.222153 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.222132    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cr27r\""
Sep 10 11:26:49.225272 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.225248    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:49.296414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.296378    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpfwc\" (UniqueName: \"kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc\") pod \"test-curl-7535-140422995671472\" (UID: \"be6ef22e-b434-4eaf-8d30-759ebafe5bf2\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:49.397150 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.397114    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpfwc\" (UniqueName: \"kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc\") pod \"test-curl-7535-140422995671472\" (UID: \"be6ef22e-b434-4eaf-8d30-759ebafe5bf2\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:49.406157 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.406113    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpfwc\" (UniqueName: \"kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc\") pod \"test-curl-7535-140422995671472\" (UID: \"be6ef22e-b434-4eaf-8d30-759ebafe5bf2\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:49.530540 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.530457    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:49.805575 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:49.805539    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:49.807163 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:26:49.807138    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe6ef22e_b434_4eaf_8d30_759ebafe5bf2.slice/crio-a1339fc14582d26f6145ba4273446c8c19d4708ff9ee4325fbbcdf4a9cb00413 WatchSource:0}: Error finding container a1339fc14582d26f6145ba4273446c8c19d4708ff9ee4325fbbcdf4a9cb00413: Status 404 returned error can't find the container with id a1339fc14582d26f6145ba4273446c8c19d4708ff9ee4325fbbcdf4a9cb00413
Sep 10 11:26:50.001509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:50.001467    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140422995671472" event={"ID":"be6ef22e-b434-4eaf-8d30-759ebafe5bf2","Type":"ContainerStarted","Data":"a1339fc14582d26f6145ba4273446c8c19d4708ff9ee4325fbbcdf4a9cb00413"}
Sep 10 11:26:50.794440 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:50.794409    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:26:52.013766 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.013737    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140422995671472" event={"ID":"be6ef22e-b434-4eaf-8d30-759ebafe5bf2","Type":"ContainerStarted","Data":"c3806a9ffb5997c221202d0fe37d24de2f39aebcf2e8ae347f2aee75f9cbaccf"}
Sep 10 11:26:52.046799 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.046742    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7535-140422995671472" podStartSLOduration=0.928183789 podStartE2EDuration="3.046721494s" totalImagesPullingTime="2.118537705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:26:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:26:52.046002504 +0000 UTC m=+746.850069852" watchObservedRunningTime="2026-09-10 11:26:52.046721494 +0000 UTC m=+746.850788843"
Sep 10 11:26:52.201126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.201040    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7535-140422995671472_be6ef22e-b434-4eaf-8d30-759ebafe5bf2/test-curl-7535-140422995671472/0.log"
Sep 10 11:26:52.825544 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.825515    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:26:52.829048 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.829028    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:52.831207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.831184    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-8355df-dockercfg-kcwls\""
Sep 10 11:26:52.831346 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.831221    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-8355df\""
Sep 10 11:26:52.839175 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.839154    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:26:52.932086 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.932057    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:52.932237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:52.932170    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fklv\" (UniqueName: \"kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.020886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.020847    2585 generic.go:356] "Generic (PLEG): container finished" podID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" containerID="c3806a9ffb5997c221202d0fe37d24de2f39aebcf2e8ae347f2aee75f9cbaccf" exitCode=0
Sep 10 11:26:53.021249 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.020935    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140422995671472" event={"ID":"be6ef22e-b434-4eaf-8d30-759ebafe5bf2","Type":"ContainerDied","Data":"c3806a9ffb5997c221202d0fe37d24de2f39aebcf2e8ae347f2aee75f9cbaccf"}
Sep 10 11:26:53.033035 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.033009    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fklv\" (UniqueName: \"kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.033087 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.033064    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.033821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.033799    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.043274 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.043247    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fklv\" (UniqueName: \"kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv\") pod \"payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") " pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.140596 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.140522    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:53.415035 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:53.415011    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:26:53.417478 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:26:53.417449    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e60f345_19a0_466e_95a9_2c09011aef8f.slice/crio-ec37898e4c14204ac8080124e9a762b83cb9be059f390233c9c9ff28032a3681 WatchSource:0}: Error finding container ec37898e4c14204ac8080124e9a762b83cb9be059f390233c9c9ff28032a3681: Status 404 returned error can't find the container with id ec37898e4c14204ac8080124e9a762b83cb9be059f390233c9c9ff28032a3681
Sep 10 11:26:54.027566 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.027532    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" event={"ID":"4e60f345-19a0-466e-95a9-2c09011aef8f","Type":"ContainerStarted","Data":"ec37898e4c14204ac8080124e9a762b83cb9be059f390233c9c9ff28032a3681"}
Sep 10 11:26:54.157701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.157671    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.188952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.188925    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:54.192976 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.192951    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:54.245692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.245661    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpfwc\" (UniqueName: \"kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc\") pod \"be6ef22e-b434-4eaf-8d30-759ebafe5bf2\" (UID: \"be6ef22e-b434-4eaf-8d30-759ebafe5bf2\") "
Sep 10 11:26:54.248345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.248292    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc" (OuterVolumeSpecName: "kube-api-access-cpfwc") pod "be6ef22e-b434-4eaf-8d30-759ebafe5bf2" (UID: "be6ef22e-b434-4eaf-8d30-759ebafe5bf2"). InnerVolumeSpecName "kube-api-access-cpfwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:26:54.333167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.333080    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:54.333553 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.333532    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" containerName="test-curl-7535-140422995671472"
Sep 10 11:26:54.333672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.333555    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" containerName="test-curl-7535-140422995671472"
Sep 10 11:26:54.333730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.333676    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" containerName="test-curl-7535-140422995671472"
Sep 10 11:26:54.336659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.336635    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.343134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.343107    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:54.346598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.346566    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpfwc\" (UniqueName: \"kubernetes.io/projected/be6ef22e-b434-4eaf-8d30-759ebafe5bf2-kube-api-access-cpfwc\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:26:54.447709 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.447673    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dd5v\" (UniqueName: \"kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v\") pod \"test-curl-7535-140422995671472\" (UID: \"87935e79-4e37-430b-9ccf-261668abf06a\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.549045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.548988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dd5v\" (UniqueName: \"kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v\") pod \"test-curl-7535-140422995671472\" (UID: \"87935e79-4e37-430b-9ccf-261668abf06a\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.558197 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.558126    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dd5v\" (UniqueName: \"kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v\") pod \"test-curl-7535-140422995671472\" (UID: \"87935e79-4e37-430b-9ccf-261668abf06a\") " pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.649667 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.649590    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:54.928879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:54.928788    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:54.932568 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:26:54.932509    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87935e79_4e37_430b_9ccf_261668abf06a.slice/crio-4f4c1a2b749506e1a915da98806dd59970de326f4f9a10b7bcc15de2f50d12f0 WatchSource:0}: Error finding container 4f4c1a2b749506e1a915da98806dd59970de326f4f9a10b7bcc15de2f50d12f0: Status 404 returned error can't find the container with id 4f4c1a2b749506e1a915da98806dd59970de326f4f9a10b7bcc15de2f50d12f0
Sep 10 11:26:55.034021 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:55.033990    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a1339fc14582d26f6145ba4273446c8c19d4708ff9ee4325fbbcdf4a9cb00413"
Sep 10 11:26:55.034487 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:55.034027    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:55.035375 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:55.035348    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140422995671472" event={"ID":"87935e79-4e37-430b-9ccf-261668abf06a","Type":"ContainerStarted","Data":"4f4c1a2b749506e1a915da98806dd59970de326f4f9a10b7bcc15de2f50d12f0"}
Sep 10 11:26:55.048195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:55.048155    2585 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7535-140422995671472" oldPodUID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" podUID="87935e79-4e37-430b-9ccf-261668abf06a"
Sep 10 11:26:55.664125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:55.664092    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be6ef22e-b434-4eaf-8d30-759ebafe5bf2" path="/var/lib/kubelet/pods/be6ef22e-b434-4eaf-8d30-759ebafe5bf2/volumes"
Sep 10 11:26:56.040991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:56.040902    2585 generic.go:356] "Generic (PLEG): container finished" podID="87935e79-4e37-430b-9ccf-261668abf06a" containerID="ddf01028cc5c0ca4bd824b5a700edf1c5a8e67bc8ab88eeaf3d4539cc53febf1" exitCode=0
Sep 10 11:26:56.040991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:56.040984    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7535-140422995671472" event={"ID":"87935e79-4e37-430b-9ccf-261668abf06a","Type":"ContainerDied","Data":"ddf01028cc5c0ca4bd824b5a700edf1c5a8e67bc8ab88eeaf3d4539cc53febf1"}
Sep 10 11:26:56.042565 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:56.042539    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" event={"ID":"4e60f345-19a0-466e-95a9-2c09011aef8f","Type":"ContainerStarted","Data":"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"}
Sep 10 11:26:56.042689 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:56.042599    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:26:56.078441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:56.078391    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" podStartSLOduration=2.159563571 podStartE2EDuration="4.078375691s" totalImagesPullingTime="1.91881212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:26:56.076538331 +0000 UTC m=+750.880605679" watchObservedRunningTime="2026-09-10 11:26:56.078375691 +0000 UTC m=+750.882443038"
Sep 10 11:26:57.175967 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.175943    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:26:57.197899 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.197867    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7535-140422995671472_87935e79-4e37-430b-9ccf-261668abf06a/test-curl-7535-140422995671472/0.log"
Sep 10 11:26:57.222516 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.222485    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:57.233982 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.233950    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7535-140422995671472"]
Sep 10 11:26:57.274602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.274574    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dd5v\" (UniqueName: \"kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v\") pod \"87935e79-4e37-430b-9ccf-261668abf06a\" (UID: \"87935e79-4e37-430b-9ccf-261668abf06a\") "
Sep 10 11:26:57.276640 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.276616    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v" (OuterVolumeSpecName: "kube-api-access-8dd5v") pod "87935e79-4e37-430b-9ccf-261668abf06a" (UID: "87935e79-4e37-430b-9ccf-261668abf06a"). InnerVolumeSpecName "kube-api-access-8dd5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:26:57.376215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.376134    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dd5v\" (UniqueName: \"kubernetes.io/projected/87935e79-4e37-430b-9ccf-261668abf06a-kube-api-access-8dd5v\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:26:57.663368 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:57.663334    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87935e79-4e37-430b-9ccf-261668abf06a" path="/var/lib/kubelet/pods/87935e79-4e37-430b-9ccf-261668abf06a/volumes"
Sep 10 11:26:58.053389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:58.053294    2585 scope.go:117] "RemoveContainer" containerID="ddf01028cc5c0ca4bd824b5a700edf1c5a8e67bc8ab88eeaf3d4539cc53febf1"
Sep 10 11:26:58.053557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:26:58.053316    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7535-140422995671472"
Sep 10 11:27:01.339449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.339410    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:01.339822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.339780    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87935e79-4e37-430b-9ccf-261668abf06a" containerName="test-curl-7535-140422995671472"
Sep 10 11:27:01.339822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.339792    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="87935e79-4e37-430b-9ccf-261668abf06a" containerName="test-curl-7535-140422995671472"
Sep 10 11:27:01.339914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.339857    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="87935e79-4e37-430b-9ccf-261668abf06a" containerName="test-curl-7535-140422995671472"
Sep 10 11:27:01.343957 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.343937    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.346193 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.346168    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-aff377da\""
Sep 10 11:27:01.346701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.346681    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-aff377da-dockercfg-tb59h\""
Sep 10 11:27:01.355158 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.355137    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:01.415401 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.415366    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kqm8\" (UniqueName: \"kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.415572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.415421    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.516723 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.516672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kqm8\" (UniqueName: \"kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.516909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.516741    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.517480 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.517461    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.527348 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.527284    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kqm8\" (UniqueName: \"kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8\") pod \"payload-processing-e2e-ait-aff377da-6476746869-jvkp2\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") " pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.653907 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.653831    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:01.940898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:01.940875    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:01.943050 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:01.943017    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4dcada6_4b4f_4f43_9781_fbdfc35d77b8.slice/crio-67a4b1516df72de6b9e5a4f63c081b99911cbb9ba03bef8240ad48bb2adc656f WatchSource:0}: Error finding container 67a4b1516df72de6b9e5a4f63c081b99911cbb9ba03bef8240ad48bb2adc656f: Status 404 returned error can't find the container with id 67a4b1516df72de6b9e5a4f63c081b99911cbb9ba03bef8240ad48bb2adc656f
Sep 10 11:27:02.074060 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:02.074022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" event={"ID":"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8","Type":"ContainerStarted","Data":"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"}
Sep 10 11:27:02.074244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:02.074100    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:02.074244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:02.074123    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" event={"ID":"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8","Type":"ContainerStarted","Data":"67a4b1516df72de6b9e5a4f63c081b99911cbb9ba03bef8240ad48bb2adc656f"}
Sep 10 11:27:02.096886 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:02.096838    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" podStartSLOduration=1.096822162 podStartE2EDuration="1.096822162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:02.094732027 +0000 UTC m=+756.898799376" watchObservedRunningTime="2026-09-10 11:27:02.096822162 +0000 UTC m=+756.900889509"
Sep 10 11:27:02.195377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:02.195296    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:27:03.720139 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.720100    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:27:03.723626 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.723609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.726863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.726844    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8de26b\""
Sep 10 11:27:03.727457 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.727437    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-8de26b-dockercfg-vxkrv\""
Sep 10 11:27:03.767246 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.767212    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:27:03.798059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.798032    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:27:03.801717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.801698    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:03.824592 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.824573    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:27:03.836034 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.836012    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h786t\" (UniqueName: \"kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.836109 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.836095    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.937146 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.937119    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h786t\" (UniqueName: \"kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.937288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.937154    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:03.937288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.937181    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7f9t\" (UniqueName: \"kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:03.937288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.937262    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.937923 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.937900    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:03.945533 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:03.945508    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h786t\" (UniqueName: \"kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t\") pod \"payload-processing-e2e-autores-8de26b-97dd787f9-cklz2\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") " pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:04.033536 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.033458    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:04.037792 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.037766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:04.037848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.037826    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7f9t\" (UniqueName: \"kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:04.038448 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.038430    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:04.046705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.046684    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7f9t\" (UniqueName: \"kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t\") pod \"payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:04.111773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.111738    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:04.307902 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.307875    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:27:04.310531 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:04.310499    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206574c2_4bd5_4e5e_8261_3061116d3740.slice/crio-db2e64f0395fa46b1576573685fe647842275ad9b53f019d731c86999b97824b WatchSource:0}: Error finding container db2e64f0395fa46b1576573685fe647842275ad9b53f019d731c86999b97824b: Status 404 returned error can't find the container with id db2e64f0395fa46b1576573685fe647842275ad9b53f019d731c86999b97824b
Sep 10 11:27:04.409662 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:04.409506    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:27:04.412338 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:04.412292    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac2e692_a221_4297_a471_7429b5383f37.slice/crio-a1784116d470dd60d561043d542a3951418bc3c78631c21d29c46a7b324932a9 WatchSource:0}: Error finding container a1784116d470dd60d561043d542a3951418bc3c78631c21d29c46a7b324932a9: Status 404 returned error can't find the container with id a1784116d470dd60d561043d542a3951418bc3c78631c21d29c46a7b324932a9
Sep 10 11:27:05.094080 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.094041    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" event={"ID":"206574c2-4bd5-4e5e-8261-3061116d3740","Type":"ContainerStarted","Data":"906106b935b96380dac7e72f4eca7fea8cf1f772c18ae0eba97723f3d80eff32"}
Sep 10 11:27:05.094557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.094106    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" event={"ID":"206574c2-4bd5-4e5e-8261-3061116d3740","Type":"ContainerStarted","Data":"db2e64f0395fa46b1576573685fe647842275ad9b53f019d731c86999b97824b"}
Sep 10 11:27:05.094557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.094157    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:05.095854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.095831    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" event={"ID":"dac2e692-a221-4297-a471-7429b5383f37","Type":"ContainerStarted","Data":"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"}
Sep 10 11:27:05.095987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.095865    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" event={"ID":"dac2e692-a221-4297-a471-7429b5383f37","Type":"ContainerStarted","Data":"a1784116d470dd60d561043d542a3951418bc3c78631c21d29c46a7b324932a9"}
Sep 10 11:27:05.095987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.095957    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:05.112355 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.112293    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" podStartSLOduration=2.112275984 podStartE2EDuration="2.112275984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:05.11125804 +0000 UTC m=+759.915325388" watchObservedRunningTime="2026-09-10 11:27:05.112275984 +0000 UTC m=+759.916343333"
Sep 10 11:27:05.130694 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:05.130649    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" podStartSLOduration=2.130632566 podStartE2EDuration="2.130632566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:05.128578761 +0000 UTC m=+759.932646108" watchObservedRunningTime="2026-09-10 11:27:05.130632566 +0000 UTC m=+759.934699918"
Sep 10 11:27:07.050066 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:07.050034    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:27:13.081646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:13.081615    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:16.104879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:16.104848    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:27:16.105365 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:16.104927    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:27:18.883688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:18.883652    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:27:22.211783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.211738    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:27:22.215199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.215183    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.218097 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.218078    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-4c9990-dockercfg-fp2cs\""
Sep 10 11:27:22.218461 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.218447    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-4c9990\""
Sep 10 11:27:22.236818 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.236794    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:27:22.306734 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.306702    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.306892 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.306759    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t26s8\" (UniqueName: \"kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.323978 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.323947    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:27:22.329942 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.329925    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.339725 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.339695    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:27:22.407815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.407789    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t26s8\" (UniqueName: \"kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.407931 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.407840    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbxpv\" (UniqueName: \"kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.407931 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.407859    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.407931 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.407877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.408505 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.408488    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.423756 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.423727    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t26s8\" (UniqueName: \"kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8\") pod \"payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.509085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.509017    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.509225 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.509133    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbxpv\" (UniqueName: \"kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.509799 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.509772    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.517280 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.517259    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbxpv\" (UniqueName: \"kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv\") pod \"payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.525075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.525053    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:22.641647 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.641619    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:22.799389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.799364    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:27:22.801554 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:22.801531    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode552131e_6679_43a3_a3fa_e0263ddd8ff8.slice/crio-798b1456c38f11cdce147bafedc35c4bc7f239984c3348a09ac0932cafc6e1cf WatchSource:0}: Error finding container 798b1456c38f11cdce147bafedc35c4bc7f239984c3348a09ac0932cafc6e1cf: Status 404 returned error can't find the container with id 798b1456c38f11cdce147bafedc35c4bc7f239984c3348a09ac0932cafc6e1cf
Sep 10 11:27:22.914594 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:22.914518    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:27:22.918874 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:22.918846    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39ec164_fe55_467b_93e0_9d130224ecc9.slice/crio-903dd6e72ee4938b79a80754fefa8bf4aab5d929bae6ba53ea944f4317645e33 WatchSource:0}: Error finding container 903dd6e72ee4938b79a80754fefa8bf4aab5d929bae6ba53ea944f4317645e33: Status 404 returned error can't find the container with id 903dd6e72ee4938b79a80754fefa8bf4aab5d929bae6ba53ea944f4317645e33
Sep 10 11:27:23.187012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.186976    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" event={"ID":"a39ec164-fe55-467b-93e0-9d130224ecc9","Type":"ContainerStarted","Data":"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"}
Sep 10 11:27:23.187188 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.187035    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" event={"ID":"a39ec164-fe55-467b-93e0-9d130224ecc9","Type":"ContainerStarted","Data":"903dd6e72ee4938b79a80754fefa8bf4aab5d929bae6ba53ea944f4317645e33"}
Sep 10 11:27:23.187188 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.187079    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:23.188619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.188594    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" event={"ID":"e552131e-6679-43a3-a3fa-e0263ddd8ff8","Type":"ContainerStarted","Data":"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"}
Sep 10 11:27:23.188722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.188638    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:23.188722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.188661    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" event={"ID":"e552131e-6679-43a3-a3fa-e0263ddd8ff8","Type":"ContainerStarted","Data":"798b1456c38f11cdce147bafedc35c4bc7f239984c3348a09ac0932cafc6e1cf"}
Sep 10 11:27:23.206034 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.205982    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" podStartSLOduration=1.205968935 podStartE2EDuration="1.205968935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:23.203829607 +0000 UTC m=+778.007896954" watchObservedRunningTime="2026-09-10 11:27:23.205968935 +0000 UTC m=+778.010036274"
Sep 10 11:27:23.224465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:23.224420    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" podStartSLOduration=1.22440351 podStartE2EDuration="1.22440351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:23.221692192 +0000 UTC m=+778.025759551" watchObservedRunningTime="2026-09-10 11:27:23.22440351 +0000 UTC m=+778.028470859"
Sep 10 11:27:24.618857 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.618826    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"]
Sep 10 11:27:24.622933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.622912    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:24.624979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.624958    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:27:24.625069 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.625047    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:27:24.629291 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.629238    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"]
Sep 10 11:27:24.730372 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.730335    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aff377da-52dzl\" (UID: \"314f6fb3-baf6-4d00-9ad3-665376a35e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:24.831452 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.831424    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aff377da-52dzl\" (UID: \"314f6fb3-baf6-4d00-9ad3-665376a35e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:24.832040 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.832016    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-aff377da-52dzl\" (UID: \"314f6fb3-baf6-4d00-9ad3-665376a35e11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:24.937289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:24.937262    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:25.207530 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:25.207503    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"]
Sep 10 11:27:25.210257 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:25.210230    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314f6fb3_baf6_4d00_9ad3_665376a35e11.slice/crio-86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd WatchSource:0}: Error finding container 86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd: Status 404 returned error can't find the container with id 86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd
Sep 10 11:27:26.206852 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:26.206816    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl" event={"ID":"314f6fb3-baf6-4d00-9ad3-665376a35e11","Type":"ContainerStarted","Data":"86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd"}
Sep 10 11:27:28.218833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:28.218801    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl" event={"ID":"314f6fb3-baf6-4d00-9ad3-665376a35e11","Type":"ContainerStarted","Data":"4584f7200e1f51ec76a7db5ab477257a9d7ed799c188b9e1e79abae6b0202c69"}
Sep 10 11:27:28.234781 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:28.234737    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl" podStartSLOduration=1.809673791 podStartE2EDuration="4.234721972s" totalImagesPullingTime="2.425048181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:28.233527283 +0000 UTC m=+783.037594632" watchObservedRunningTime="2026-09-10 11:27:28.234721972 +0000 UTC m=+783.038789364"
Sep 10 11:27:29.225751 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:29.225718    2585 generic.go:356] "Generic (PLEG): container finished" podID="314f6fb3-baf6-4d00-9ad3-665376a35e11" containerID="4584f7200e1f51ec76a7db5ab477257a9d7ed799c188b9e1e79abae6b0202c69" exitCode=0
Sep 10 11:27:29.226122 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:29.225763    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl" event={"ID":"314f6fb3-baf6-4d00-9ad3-665376a35e11","Type":"ContainerDied","Data":"4584f7200e1f51ec76a7db5ab477257a9d7ed799c188b9e1e79abae6b0202c69"}
Sep 10 11:27:30.359010 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:30.358982    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:30.480100 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:30.480070    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca\") pod \"314f6fb3-baf6-4d00-9ad3-665376a35e11\" (UID: \"314f6fb3-baf6-4d00-9ad3-665376a35e11\") "
Sep 10 11:27:30.480468 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:30.480442    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "314f6fb3-baf6-4d00-9ad3-665376a35e11" (UID: "314f6fb3-baf6-4d00-9ad3-665376a35e11"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:27:30.581258 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:30.581191    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/314f6fb3-baf6-4d00-9ad3-665376a35e11-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:27:31.237777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:31.237747    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl" event={"ID":"314f6fb3-baf6-4d00-9ad3-665376a35e11","Type":"ContainerDied","Data":"86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd"}
Sep 10 11:27:31.237777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:31.237778    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"
Sep 10 11:27:31.238045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:31.237793    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86bc4b3f28224e773971ae4fe426d66ecb7b9a96399b28205667dccaf7749efd"
Sep 10 11:27:33.861399 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:33.861361    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:27:34.196321 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:34.196269    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:27:34.196500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:34.196358    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:27:41.383714 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.383681    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:41.384171 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.383974    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" podUID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" containerName="payload-processing" containerID="cri-o://298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61" gracePeriod=30
Sep 10 11:27:41.639372 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.639272    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:41.676789 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.676761    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"]
Sep 10 11:27:41.695718 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.695692    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-aff377da-52dzl"]
Sep 10 11:27:41.781485 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.781451    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kqm8\" (UniqueName: \"kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8\") pod \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") "
Sep 10 11:27:41.781624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.781504    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume\") pod \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\" (UID: \"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8\") "
Sep 10 11:27:41.781933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.781907    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" (UID: "a4dcada6-4b4f-4f43-9781-fbdfc35d77b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:27:41.783831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.783810    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8" (OuterVolumeSpecName: "kube-api-access-4kqm8") pod "a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" (UID: "a4dcada6-4b4f-4f43-9781-fbdfc35d77b8"). InnerVolumeSpecName "kube-api-access-4kqm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:27:41.882685 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.882648    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kqm8\" (UniqueName: \"kubernetes.io/projected/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-kube-api-access-4kqm8\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:27:41.882685 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:41.882678    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:27:42.295424 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.295380    2585 generic.go:356] "Generic (PLEG): container finished" podID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" containerID="298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61" exitCode=0
Sep 10 11:27:42.295584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.295432    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"
Sep 10 11:27:42.295584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.295463    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" event={"ID":"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8","Type":"ContainerDied","Data":"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"}
Sep 10 11:27:42.295584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.295521    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2" event={"ID":"a4dcada6-4b4f-4f43-9781-fbdfc35d77b8","Type":"ContainerDied","Data":"67a4b1516df72de6b9e5a4f63c081b99911cbb9ba03bef8240ad48bb2adc656f"}
Sep 10 11:27:42.295584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.295544    2585 scope.go:117] "RemoveContainer" containerID="298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"
Sep 10 11:27:42.304713 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.304693    2585 scope.go:117] "RemoveContainer" containerID="298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"
Sep 10 11:27:42.304990 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:27:42.304969    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61\": container with ID starting with 298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61 not found: ID does not exist" containerID="298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"
Sep 10 11:27:42.305033 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.304999    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61"} err="failed to get container status \"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61\": rpc error: code = NotFound desc = could not find container \"298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61\": container with ID starting with 298b0c9911dac57d577cf80d8dd1aab9928be1eccaf125a25b755dd22a767e61 not found: ID does not exist"
Sep 10 11:27:42.336362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.336327    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:42.340123 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:42.340101    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-aff377da-6476746869-jvkp2"]
Sep 10 11:27:43.663033 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:43.662998    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="314f6fb3-baf6-4d00-9ad3-665376a35e11" path="/var/lib/kubelet/pods/314f6fb3-baf6-4d00-9ad3-665376a35e11/volumes"
Sep 10 11:27:43.663559 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:43.663536    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" path="/var/lib/kubelet/pods/a4dcada6-4b4f-4f43-9781-fbdfc35d77b8/volumes"
Sep 10 11:27:46.651089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651059    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:27:46.651488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651458    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" containerName="payload-processing"
Sep 10 11:27:46.651488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651471    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" containerName="payload-processing"
Sep 10 11:27:46.651488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651489    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="314f6fb3-baf6-4d00-9ad3-665376a35e11" containerName="revoke-keys"
Sep 10 11:27:46.651581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651494    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="314f6fb3-baf6-4d00-9ad3-665376a35e11" containerName="revoke-keys"
Sep 10 11:27:46.651581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651555    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4dcada6-4b4f-4f43-9781-fbdfc35d77b8" containerName="payload-processing"
Sep 10 11:27:46.651581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.651565    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="314f6fb3-baf6-4d00-9ad3-665376a35e11" containerName="revoke-keys"
Sep 10 11:27:46.655880 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.655863    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.658001 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.657981    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-913c8e9c\""
Sep 10 11:27:46.658215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.658198    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-913c8e9c-dockercfg-fvg8b\""
Sep 10 11:27:46.667502 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.667479    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:27:46.726817 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.726786    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpvp9\" (UniqueName: \"kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.727039 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.727015    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.794166 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.794132    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:27:46.797592 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.797577    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:46.810253 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.810229    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:27:46.827538 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.827516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.827626 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.827568    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpvp9\" (UniqueName: \"kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.828230 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.828212    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.839374 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.839347    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpvp9\" (UniqueName: \"kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9\") pod \"payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:46.928829 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.928791    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:46.928999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.928862    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph7nb\" (UniqueName: \"kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:46.966771 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:46.966729    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:47.030262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.030224    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:47.030510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.030362    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph7nb\" (UniqueName: \"kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:47.032173 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.032080    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:47.045690 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.045668    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph7nb\" (UniqueName: \"kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb\") pod \"payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") " pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:47.108272 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.108238    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:47.242161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.242121    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:27:47.246712 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:47.246682    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0377168_9887_402c_ab28_cfd9875f4e91.slice/crio-7acfd8b92fd3b6e41523551f285c8672ac98bce46546e3546d7495052363716c WatchSource:0}: Error finding container 7acfd8b92fd3b6e41523551f285c8672ac98bce46546e3546d7495052363716c: Status 404 returned error can't find the container with id 7acfd8b92fd3b6e41523551f285c8672ac98bce46546e3546d7495052363716c
Sep 10 11:27:47.323836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.323803    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" event={"ID":"a0377168-9887-402c-ab28-cfd9875f4e91","Type":"ContainerStarted","Data":"9095000d14d6e3ee3a413b7c7ba45eff324b4c67a22b741210958464b5dab15d"}
Sep 10 11:27:47.323960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.323855    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" event={"ID":"a0377168-9887-402c-ab28-cfd9875f4e91","Type":"ContainerStarted","Data":"7acfd8b92fd3b6e41523551f285c8672ac98bce46546e3546d7495052363716c"}
Sep 10 11:27:47.323960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.323925    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:47.343933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.343891    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" podStartSLOduration=1.343873388 podStartE2EDuration="1.343873388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:47.342460057 +0000 UTC m=+802.146527406" watchObservedRunningTime="2026-09-10 11:27:47.343873388 +0000 UTC m=+802.147940737"
Sep 10 11:27:47.448940 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:47.448907    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:27:47.450677 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:27:47.450655    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c34756_4d16_4eaa_82bb_9b411acdbee7.slice/crio-221530d1290ebe02972c49e4d6ad79e1643cfc7978b08b111d701635f2b68364 WatchSource:0}: Error finding container 221530d1290ebe02972c49e4d6ad79e1643cfc7978b08b111d701635f2b68364: Status 404 returned error can't find the container with id 221530d1290ebe02972c49e4d6ad79e1643cfc7978b08b111d701635f2b68364
Sep 10 11:27:48.330441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:48.330236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" event={"ID":"33c34756-4d16-4eaa-82bb-9b411acdbee7","Type":"ContainerStarted","Data":"2144b9c3b7e2f4f1468b729f690718af7e209d3b68ccce4bc802059875ba1525"}
Sep 10 11:27:48.330441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:48.330292    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" event={"ID":"33c34756-4d16-4eaa-82bb-9b411acdbee7","Type":"ContainerStarted","Data":"221530d1290ebe02972c49e4d6ad79e1643cfc7978b08b111d701635f2b68364"}
Sep 10 11:27:48.330441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:48.330333    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:27:48.349834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:48.349790    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" podStartSLOduration=2.349774315 podStartE2EDuration="2.349774315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:27:48.34819437 +0000 UTC m=+803.152261717" watchObservedRunningTime="2026-09-10 11:27:48.349774315 +0000 UTC m=+803.153841684"
Sep 10 11:27:48.578991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:48.578960    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:27:58.332428 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:58.332397    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:27:59.338670 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:27:59.338638    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:28:05.373020 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:05.372967    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:28:15.861678 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:15.861637    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:28:17.249952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.249916    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:28:17.250406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.250145    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" podUID="33c34756-4d16-4eaa-82bb-9b411acdbee7" containerName="payload-processing" containerID="cri-o://2144b9c3b7e2f4f1468b729f690718af7e209d3b68ccce4bc802059875ba1525" gracePeriod=30
Sep 10 11:28:17.271231 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.271207    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:28:17.271510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.271451    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" podUID="a0377168-9887-402c-ab28-cfd9875f4e91" containerName="payload-pre-processing" containerID="cri-o://9095000d14d6e3ee3a413b7c7ba45eff324b4c67a22b741210958464b5dab15d" gracePeriod=30
Sep 10 11:28:17.479706 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.479675    2585 generic.go:356] "Generic (PLEG): container finished" podID="33c34756-4d16-4eaa-82bb-9b411acdbee7" containerID="2144b9c3b7e2f4f1468b729f690718af7e209d3b68ccce4bc802059875ba1525" exitCode=0
Sep 10 11:28:17.479860 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.479752    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" event={"ID":"33c34756-4d16-4eaa-82bb-9b411acdbee7","Type":"ContainerDied","Data":"2144b9c3b7e2f4f1468b729f690718af7e209d3b68ccce4bc802059875ba1525"}
Sep 10 11:28:17.481618 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.481596    2585 generic.go:356] "Generic (PLEG): container finished" podID="a0377168-9887-402c-ab28-cfd9875f4e91" containerID="9095000d14d6e3ee3a413b7c7ba45eff324b4c67a22b741210958464b5dab15d" exitCode=0
Sep 10 11:28:17.481743 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.481664    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" event={"ID":"a0377168-9887-402c-ab28-cfd9875f4e91","Type":"ContainerDied","Data":"9095000d14d6e3ee3a413b7c7ba45eff324b4c67a22b741210958464b5dab15d"}
Sep 10 11:28:17.547823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.547794    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:28:17.550683 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.550664    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:28:17.599499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.599473    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume\") pod \"33c34756-4d16-4eaa-82bb-9b411acdbee7\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") "
Sep 10 11:28:17.599668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.599570    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpvp9\" (UniqueName: \"kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9\") pod \"a0377168-9887-402c-ab28-cfd9875f4e91\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") "
Sep 10 11:28:17.599668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.599621    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph7nb\" (UniqueName: \"kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb\") pod \"33c34756-4d16-4eaa-82bb-9b411acdbee7\" (UID: \"33c34756-4d16-4eaa-82bb-9b411acdbee7\") "
Sep 10 11:28:17.599668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.599649    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume\") pod \"a0377168-9887-402c-ab28-cfd9875f4e91\" (UID: \"a0377168-9887-402c-ab28-cfd9875f4e91\") "
Sep 10 11:28:17.599911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.599877    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "33c34756-4d16-4eaa-82bb-9b411acdbee7" (UID: "33c34756-4d16-4eaa-82bb-9b411acdbee7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:28:17.600330 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.600283    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0377168-9887-402c-ab28-cfd9875f4e91" (UID: "a0377168-9887-402c-ab28-cfd9875f4e91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:28:17.602126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.602097    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9" (OuterVolumeSpecName: "kube-api-access-mpvp9") pod "a0377168-9887-402c-ab28-cfd9875f4e91" (UID: "a0377168-9887-402c-ab28-cfd9875f4e91"). InnerVolumeSpecName "kube-api-access-mpvp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:28:17.602494 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.602469    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb" (OuterVolumeSpecName: "kube-api-access-ph7nb") pod "33c34756-4d16-4eaa-82bb-9b411acdbee7" (UID: "33c34756-4d16-4eaa-82bb-9b411acdbee7"). InnerVolumeSpecName "kube-api-access-ph7nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:28:17.701212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.701177    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/33c34756-4d16-4eaa-82bb-9b411acdbee7-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:17.701212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.701213    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpvp9\" (UniqueName: \"kubernetes.io/projected/a0377168-9887-402c-ab28-cfd9875f4e91-kube-api-access-mpvp9\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:17.701453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.701230    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph7nb\" (UniqueName: \"kubernetes.io/projected/33c34756-4d16-4eaa-82bb-9b411acdbee7-kube-api-access-ph7nb\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:17.701453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:17.701244    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0377168-9887-402c-ab28-cfd9875f4e91-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:18.488947 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.488915    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"
Sep 10 11:28:18.488947 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.488934    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l" event={"ID":"33c34756-4d16-4eaa-82bb-9b411acdbee7","Type":"ContainerDied","Data":"221530d1290ebe02972c49e4d6ad79e1643cfc7978b08b111d701635f2b68364"}
Sep 10 11:28:18.489470 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.488987    2585 scope.go:117] "RemoveContainer" containerID="2144b9c3b7e2f4f1468b729f690718af7e209d3b68ccce4bc802059875ba1525"
Sep 10 11:28:18.490334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.490317    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"
Sep 10 11:28:18.490440 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.490339    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n" event={"ID":"a0377168-9887-402c-ab28-cfd9875f4e91","Type":"ContainerDied","Data":"7acfd8b92fd3b6e41523551f285c8672ac98bce46546e3546d7495052363716c"}
Sep 10 11:28:18.498409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.498379    2585 scope.go:117] "RemoveContainer" containerID="9095000d14d6e3ee3a413b7c7ba45eff324b4c67a22b741210958464b5dab15d"
Sep 10 11:28:18.507205 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.507180    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:28:18.510807 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.510784    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-913c8e9c-649bdbfddb-htj7n"]
Sep 10 11:28:18.523832 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.523788    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:28:18.530393 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:18.530375    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-913c8e9c-6f78c859d6-2ds5l"]
Sep 10 11:28:19.663203 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:19.663170    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c34756-4d16-4eaa-82bb-9b411acdbee7" path="/var/lib/kubelet/pods/33c34756-4d16-4eaa-82bb-9b411acdbee7/volumes"
Sep 10 11:28:19.663594 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:19.663547    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0377168-9887-402c-ab28-cfd9875f4e91" path="/var/lib/kubelet/pods/a0377168-9887-402c-ab28-cfd9875f4e91/volumes"
Sep 10 11:28:22.765391 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765357    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:22.765779 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765764    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0377168-9887-402c-ab28-cfd9875f4e91" containerName="payload-pre-processing"
Sep 10 11:28:22.765823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765780    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0377168-9887-402c-ab28-cfd9875f4e91" containerName="payload-pre-processing"
Sep 10 11:28:22.765823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765793    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c34756-4d16-4eaa-82bb-9b411acdbee7" containerName="payload-processing"
Sep 10 11:28:22.765823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765798    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c34756-4d16-4eaa-82bb-9b411acdbee7" containerName="payload-processing"
Sep 10 11:28:22.765915 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765864    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0377168-9887-402c-ab28-cfd9875f4e91" containerName="payload-pre-processing"
Sep 10 11:28:22.765915 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.765872    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c34756-4d16-4eaa-82bb-9b411acdbee7" containerName="payload-processing"
Sep 10 11:28:22.768834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.768819    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.771004 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.770981    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-54215d39\""
Sep 10 11:28:22.784485 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.784464    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:22.838791 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.838764    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:22.841970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.841955    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:22.844482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.844460    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-54215d39-dockercfg-5scmn\""
Sep 10 11:28:22.848746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.848722    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q229\" (UniqueName: \"kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.848830 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.848817    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.852841 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.852819    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:22.949864 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.949834    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q229\" (UniqueName: \"kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.950025 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.949878    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c774\" (UniqueName: \"kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:22.950025 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.950001    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:22.950118 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.950037    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.950697 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.950680    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:22.958176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:22.958155    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q229\" (UniqueName: \"kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229\") pod \"payload-processing-e2e-ait-54215d39-7645f59885-74thh\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") " pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:23.050753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.050672    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c774\" (UniqueName: \"kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.050753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.050754    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.051436 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.051416    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.058900 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.058875    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c774\" (UniqueName: \"kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774\") pod \"payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.078878 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.078854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:23.151715 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.151684    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.353345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.353314    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:23.355239 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:28:23.355212    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05a66212_b4d7_4c1b_8580_355ba2011a06.slice/crio-52ba0e39e0e899266d88726c94da6368ece81d43ebcb236c665e628906554520 WatchSource:0}: Error finding container 52ba0e39e0e899266d88726c94da6368ece81d43ebcb236c665e628906554520: Status 404 returned error can't find the container with id 52ba0e39e0e899266d88726c94da6368ece81d43ebcb236c665e628906554520
Sep 10 11:28:23.423463 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.423436    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:23.424776 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:28:23.424752    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac4fdcc_0bac_4f16_b307_7de9a4b314b0.slice/crio-0ebe53a47dad5f294470a3225cf42d1b54aa5c2074827b2996a1f91546ac379b WatchSource:0}: Error finding container 0ebe53a47dad5f294470a3225cf42d1b54aa5c2074827b2996a1f91546ac379b: Status 404 returned error can't find the container with id 0ebe53a47dad5f294470a3225cf42d1b54aa5c2074827b2996a1f91546ac379b
Sep 10 11:28:23.519999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.519968    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" event={"ID":"aac4fdcc-0bac-4f16-b307-7de9a4b314b0","Type":"ContainerStarted","Data":"9e45897b3a5b56e78c5f2707e98fd6c3da3ef84f56d8afb77879c9ed8b88a3bc"}
Sep 10 11:28:23.520135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.520021    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" event={"ID":"aac4fdcc-0bac-4f16-b307-7de9a4b314b0","Type":"ContainerStarted","Data":"0ebe53a47dad5f294470a3225cf42d1b54aa5c2074827b2996a1f91546ac379b"}
Sep 10 11:28:23.520135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.520070    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:23.521875 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.521854    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" event={"ID":"05a66212-b4d7-4c1b-8580-355ba2011a06","Type":"ContainerStarted","Data":"9849a0becca53453263104346895d8d43e31997e3765ddac165afadcf6b98aab"}
Sep 10 11:28:23.521994 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.521901    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:23.521994 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.521928    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" event={"ID":"05a66212-b4d7-4c1b-8580-355ba2011a06","Type":"ContainerStarted","Data":"52ba0e39e0e899266d88726c94da6368ece81d43ebcb236c665e628906554520"}
Sep 10 11:28:23.537765 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.537725    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" podStartSLOduration=1.537710363 podStartE2EDuration="1.537710363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:28:23.536421194 +0000 UTC m=+838.340488543" watchObservedRunningTime="2026-09-10 11:28:23.537710363 +0000 UTC m=+838.341777711"
Sep 10 11:28:23.553559 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:23.553520    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" podStartSLOduration=1.553504016 podStartE2EDuration="1.553504016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:28:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:28:23.552566175 +0000 UTC m=+838.356633523" watchObservedRunningTime="2026-09-10 11:28:23.553504016 +0000 UTC m=+838.357571366"
Sep 10 11:28:33.572805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:33.572771    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:28:34.529040 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:34.529008    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:34.529233 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:34.529096    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:35.840433 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.840397    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:28:35.845684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.845662    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.847771 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.847754    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-27987e85-kserve-self-signed-certs\""
Sep 10 11:28:35.852805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.852784    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:28:35.970355 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970325    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.970490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970368    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.970490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970399    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.970490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970448    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvvs7\" (UniqueName: \"kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.970490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970476    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:35.970627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:35.970510    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.071913 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.071884    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.071926    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.071958    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.071995    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvvs7\" (UniqueName: \"kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.072043    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.072109    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072435 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.072415    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072506 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.072432    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.072561 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.072508    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.074200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.074176    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.074499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.074480    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.080979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.080957    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvvs7\" (UniqueName: \"kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7\") pod \"e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") " pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.155464 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.155389    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:36.433348 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.433318    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:28:36.434934 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:28:36.434906    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64e2a07a_1996_4701_8d87_35f61028ce61.slice/crio-b099fa9de5f1b4aa1a5fc76be9a816b17aa0bc9cc5a202dff3d3d659ecc1bb65 WatchSource:0}: Error finding container b099fa9de5f1b4aa1a5fc76be9a816b17aa0bc9cc5a202dff3d3d659ecc1bb65: Status 404 returned error can't find the container with id b099fa9de5f1b4aa1a5fc76be9a816b17aa0bc9cc5a202dff3d3d659ecc1bb65
Sep 10 11:28:36.586924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.586888    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" event={"ID":"64e2a07a-1996-4701-8d87-35f61028ce61","Type":"ContainerStarted","Data":"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"}
Sep 10 11:28:36.586924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.586931    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" event={"ID":"64e2a07a-1996-4701-8d87-35f61028ce61","Type":"ContainerStarted","Data":"b099fa9de5f1b4aa1a5fc76be9a816b17aa0bc9cc5a202dff3d3d659ecc1bb65"}
Sep 10 11:28:36.609705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:36.609662    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" podStartSLOduration=1.6096479719999999 podStartE2EDuration="1.609647972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:28:36.607387666 +0000 UTC m=+851.411455011" watchObservedRunningTime="2026-09-10 11:28:36.609647972 +0000 UTC m=+851.413715320"
Sep 10 11:28:46.155770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:46.155734    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:46.156133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:46.155788    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:46.168368 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:46.168339    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:46.649871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:46.649842    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:28:49.262752 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:49.262711    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:28:52.432852 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.432819    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:52.433340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.433056    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" podUID="05a66212-b4d7-4c1b-8580-355ba2011a06" containerName="payload-processing" containerID="cri-o://9849a0becca53453263104346895d8d43e31997e3765ddac165afadcf6b98aab" gracePeriod=30
Sep 10 11:28:52.443646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.443619    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:52.443874 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.443849    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" podUID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" containerName="payload-pre-processing" containerID="cri-o://9e45897b3a5b56e78c5f2707e98fd6c3da3ef84f56d8afb77879c9ed8b88a3bc" gracePeriod=30
Sep 10 11:28:52.672723 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.672690    2585 generic.go:356] "Generic (PLEG): container finished" podID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" containerID="9e45897b3a5b56e78c5f2707e98fd6c3da3ef84f56d8afb77879c9ed8b88a3bc" exitCode=0
Sep 10 11:28:52.672882 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.672769    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" event={"ID":"aac4fdcc-0bac-4f16-b307-7de9a4b314b0","Type":"ContainerDied","Data":"9e45897b3a5b56e78c5f2707e98fd6c3da3ef84f56d8afb77879c9ed8b88a3bc"}
Sep 10 11:28:52.674904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.674878    2585 generic.go:356] "Generic (PLEG): container finished" podID="05a66212-b4d7-4c1b-8580-355ba2011a06" containerID="9849a0becca53453263104346895d8d43e31997e3765ddac165afadcf6b98aab" exitCode=0
Sep 10 11:28:52.675001 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.674906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" event={"ID":"05a66212-b4d7-4c1b-8580-355ba2011a06","Type":"ContainerDied","Data":"9849a0becca53453263104346895d8d43e31997e3765ddac165afadcf6b98aab"}
Sep 10 11:28:52.695987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.695936    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:52.716835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.716730    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q229\" (UniqueName: \"kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229\") pod \"05a66212-b4d7-4c1b-8580-355ba2011a06\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") "
Sep 10 11:28:52.716835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.716777    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume\") pod \"05a66212-b4d7-4c1b-8580-355ba2011a06\" (UID: \"05a66212-b4d7-4c1b-8580-355ba2011a06\") "
Sep 10 11:28:52.717705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.717209    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05a66212-b4d7-4c1b-8580-355ba2011a06" (UID: "05a66212-b4d7-4c1b-8580-355ba2011a06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:28:52.719160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.719136    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229" (OuterVolumeSpecName: "kube-api-access-6q229") pod "05a66212-b4d7-4c1b-8580-355ba2011a06" (UID: "05a66212-b4d7-4c1b-8580-355ba2011a06"). InnerVolumeSpecName "kube-api-access-6q229". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:28:52.725569 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.725551    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:52.817602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.817571    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume\") pod \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") "
Sep 10 11:28:52.817724 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.817681    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c774\" (UniqueName: \"kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774\") pod \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\" (UID: \"aac4fdcc-0bac-4f16-b307-7de9a4b314b0\") "
Sep 10 11:28:52.817854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.817842    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q229\" (UniqueName: \"kubernetes.io/projected/05a66212-b4d7-4c1b-8580-355ba2011a06-kube-api-access-6q229\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:52.817893 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.817855    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05a66212-b4d7-4c1b-8580-355ba2011a06-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:52.817981 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.817955    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aac4fdcc-0bac-4f16-b307-7de9a4b314b0" (UID: "aac4fdcc-0bac-4f16-b307-7de9a4b314b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:28:52.819648 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.819630    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774" (OuterVolumeSpecName: "kube-api-access-6c774") pod "aac4fdcc-0bac-4f16-b307-7de9a4b314b0" (UID: "aac4fdcc-0bac-4f16-b307-7de9a4b314b0"). InnerVolumeSpecName "kube-api-access-6c774". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:28:52.918625 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.918595    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c774\" (UniqueName: \"kubernetes.io/projected/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-kube-api-access-6c774\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:52.918625 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:52.918621    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aac4fdcc-0bac-4f16-b307-7de9a4b314b0-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:28:53.681366 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.681331    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv" event={"ID":"aac4fdcc-0bac-4f16-b307-7de9a4b314b0","Type":"ContainerDied","Data":"0ebe53a47dad5f294470a3225cf42d1b54aa5c2074827b2996a1f91546ac379b"}
Sep 10 11:28:53.681366 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.681360    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"
Sep 10 11:28:53.681795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.681394    2585 scope.go:117] "RemoveContainer" containerID="9e45897b3a5b56e78c5f2707e98fd6c3da3ef84f56d8afb77879c9ed8b88a3bc"
Sep 10 11:28:53.682786 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.682765    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"
Sep 10 11:28:53.682786 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.682773    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh" event={"ID":"05a66212-b4d7-4c1b-8580-355ba2011a06","Type":"ContainerDied","Data":"52ba0e39e0e899266d88726c94da6368ece81d43ebcb236c665e628906554520"}
Sep 10 11:28:53.690084 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.690066    2585 scope.go:117] "RemoveContainer" containerID="9849a0becca53453263104346895d8d43e31997e3765ddac165afadcf6b98aab"
Sep 10 11:28:53.699726 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.699680    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:53.705266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.705245    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-54215d39-67b96dcf5d-qcjzv"]
Sep 10 11:28:53.715655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.715634    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:53.720173 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:53.720156    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-54215d39-7645f59885-74thh"]
Sep 10 11:28:55.663732 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:55.663699    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05a66212-b4d7-4c1b-8580-355ba2011a06" path="/var/lib/kubelet/pods/05a66212-b4d7-4c1b-8580-355ba2011a06/volumes"
Sep 10 11:28:55.664126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:55.664054    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" path="/var/lib/kubelet/pods/aac4fdcc-0bac-4f16-b307-7de9a4b314b0/volumes"
Sep 10 11:28:58.850338 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850293    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:28:58.850727 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850714    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" containerName="payload-pre-processing"
Sep 10 11:28:58.850772 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850728    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" containerName="payload-pre-processing"
Sep 10 11:28:58.850772 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850747    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05a66212-b4d7-4c1b-8580-355ba2011a06" containerName="payload-processing"
Sep 10 11:28:58.850772 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850753    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="05a66212-b4d7-4c1b-8580-355ba2011a06" containerName="payload-processing"
Sep 10 11:28:58.850908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850810    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="aac4fdcc-0bac-4f16-b307-7de9a4b314b0" containerName="payload-pre-processing"
Sep 10 11:28:58.850908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.850821    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="05a66212-b4d7-4c1b-8580-355ba2011a06" containerName="payload-processing"
Sep 10 11:28:58.855244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.855227    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.858668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.858647    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0024ffd2\""
Sep 10 11:28:58.858941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.858925    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0024ffd2-dockercfg-2snkf\""
Sep 10 11:28:58.865885 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.865866    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.865953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.865929    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-854n7\" (UniqueName: \"kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.869888 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.869867    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:28:58.925346 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.925320    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:28:58.928764 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.928750    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:58.943235 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.943212    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:28:58.966799 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.966773    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:58.966901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.966808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-854n7\" (UniqueName: \"kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.966901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.966889    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57jcp\" (UniqueName: \"kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:58.966975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.966922    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.967561 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.967545    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:58.982068 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:58.982041    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-854n7\" (UniqueName: \"kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7\") pod \"payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") " pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:59.067443 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.067412    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57jcp\" (UniqueName: \"kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.067581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.067472    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.068100 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.068082    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.075673 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.075650    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57jcp\" (UniqueName: \"kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp\") pod \"payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.165199 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.165171    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:59.244244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.243916    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.475543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.475467    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:28:59.478695 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:28:59.478657    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f12ec3_50d9_4603_94b4_43c94b7a5d06.slice/crio-c6486c56117e310b543be2258b6e35b074a5bd4556a79caa6ea5c92f62cd6f53 WatchSource:0}: Error finding container c6486c56117e310b543be2258b6e35b074a5bd4556a79caa6ea5c92f62cd6f53: Status 404 returned error can't find the container with id c6486c56117e310b543be2258b6e35b074a5bd4556a79caa6ea5c92f62cd6f53
Sep 10 11:28:59.531389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.531366    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:28:59.532586 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:28:59.532566    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod249b3e9b_4ba7_4757_b1c0_132718957d2f.slice/crio-7c3c7cc4f88f7da7d3ad8251e04c41e2868dafb351df9a3dd6b02cc2c53ba7f5 WatchSource:0}: Error finding container 7c3c7cc4f88f7da7d3ad8251e04c41e2868dafb351df9a3dd6b02cc2c53ba7f5: Status 404 returned error can't find the container with id 7c3c7cc4f88f7da7d3ad8251e04c41e2868dafb351df9a3dd6b02cc2c53ba7f5
Sep 10 11:28:59.720500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.720458    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" event={"ID":"c0f12ec3-50d9-4603-94b4-43c94b7a5d06","Type":"ContainerStarted","Data":"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"}
Sep 10 11:28:59.720788 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.720770    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" event={"ID":"c0f12ec3-50d9-4603-94b4-43c94b7a5d06","Type":"ContainerStarted","Data":"c6486c56117e310b543be2258b6e35b074a5bd4556a79caa6ea5c92f62cd6f53"}
Sep 10 11:28:59.720948 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.720935    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:28:59.723081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.722931    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" event={"ID":"249b3e9b-4ba7-4757-b1c0-132718957d2f","Type":"ContainerStarted","Data":"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"}
Sep 10 11:28:59.723081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.722987    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" event={"ID":"249b3e9b-4ba7-4757-b1c0-132718957d2f","Type":"ContainerStarted","Data":"7c3c7cc4f88f7da7d3ad8251e04c41e2868dafb351df9a3dd6b02cc2c53ba7f5"}
Sep 10 11:28:59.723698 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.723679    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:28:59.759650 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.759525    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" podStartSLOduration=1.7595059389999999 podStartE2EDuration="1.759505939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:28:59.758677935 +0000 UTC m=+874.562745285" watchObservedRunningTime="2026-09-10 11:28:59.759505939 +0000 UTC m=+874.563573287"
Sep 10 11:28:59.788952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:28:59.788883    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" podStartSLOduration=1.788862558 podStartE2EDuration="1.788862558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:28:59.785881613 +0000 UTC m=+874.589948960" watchObservedRunningTime="2026-09-10 11:28:59.788862558 +0000 UTC m=+874.592929909"
Sep 10 11:29:02.809795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:02.809761    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:29:10.731994 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:10.731956    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:29:10.732478 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:10.732033    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:29:16.519075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:16.519041    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:29:20.826216 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:20.826129    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:29:25.643031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:25.642991    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:29:25.643489 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:25.643453    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:29:28.464598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.464557    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:29:28.465085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.464825    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" podUID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" containerName="payload-processing" containerID="cri-o://3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8" gracePeriod=30
Sep 10 11:29:28.473577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.473550    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:29:28.473790 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.473766    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" podUID="249b3e9b-4ba7-4757-b1c0-132718957d2f" containerName="payload-pre-processing" containerID="cri-o://e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29" gracePeriod=30
Sep 10 11:29:28.678917 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.678882    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:29:28.679125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.679098    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" podUID="206574c2-4bd5-4e5e-8261-3061116d3740" containerName="payload-processing" containerID="cri-o://906106b935b96380dac7e72f4eca7fea8cf1f772c18ae0eba97723f3d80eff32" gracePeriod=30
Sep 10 11:29:28.701528 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.692819    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:29:28.790967 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.790942    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:29:28.794456 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.794436    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:29:28.874659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.874623    2585 generic.go:356] "Generic (PLEG): container finished" podID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" containerID="3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8" exitCode=0
Sep 10 11:29:28.874813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.874671    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"
Sep 10 11:29:28.874813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.874703    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" event={"ID":"c0f12ec3-50d9-4603-94b4-43c94b7a5d06","Type":"ContainerDied","Data":"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"}
Sep 10 11:29:28.874813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.874766    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv" event={"ID":"c0f12ec3-50d9-4603-94b4-43c94b7a5d06","Type":"ContainerDied","Data":"c6486c56117e310b543be2258b6e35b074a5bd4556a79caa6ea5c92f62cd6f53"}
Sep 10 11:29:28.874813 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.874796    2585 scope.go:117] "RemoveContainer" containerID="3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"
Sep 10 11:29:28.876755 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.876733    2585 generic.go:356] "Generic (PLEG): container finished" podID="206574c2-4bd5-4e5e-8261-3061116d3740" containerID="906106b935b96380dac7e72f4eca7fea8cf1f772c18ae0eba97723f3d80eff32" exitCode=0
Sep 10 11:29:28.876911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.876894    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" event={"ID":"206574c2-4bd5-4e5e-8261-3061116d3740","Type":"ContainerDied","Data":"906106b935b96380dac7e72f4eca7fea8cf1f772c18ae0eba97723f3d80eff32"}
Sep 10 11:29:28.878808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.878779    2585 generic.go:356] "Generic (PLEG): container finished" podID="249b3e9b-4ba7-4757-b1c0-132718957d2f" containerID="e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29" exitCode=0
Sep 10 11:29:28.878933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.878861    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"
Sep 10 11:29:28.878933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.878909    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" event={"ID":"249b3e9b-4ba7-4757-b1c0-132718957d2f","Type":"ContainerDied","Data":"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"}
Sep 10 11:29:28.879058 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.878950    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f" event={"ID":"249b3e9b-4ba7-4757-b1c0-132718957d2f","Type":"ContainerDied","Data":"7c3c7cc4f88f7da7d3ad8251e04c41e2868dafb351df9a3dd6b02cc2c53ba7f5"}
Sep 10 11:29:28.879115 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.879065    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" podUID="dac2e692-a221-4297-a471-7429b5383f37" containerName="payload-pre-processing" containerID="cri-o://a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8" gracePeriod=30
Sep 10 11:29:28.889539 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.889520    2585 scope.go:117] "RemoveContainer" containerID="3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"
Sep 10 11:29:28.890333 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:29:28.890039    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8\": container with ID starting with 3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8 not found: ID does not exist" containerID="3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"
Sep 10 11:29:28.890333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.890073    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8"} err="failed to get container status \"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8\": rpc error: code = NotFound desc = could not find container \"3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8\": container with ID starting with 3c27225aedf92758e208ec98e4ee29b1a8114d330cf446b0c18d1a24c8e4d1b8 not found: ID does not exist"
Sep 10 11:29:28.890333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.890093    2585 scope.go:117] "RemoveContainer" containerID="e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"
Sep 10 11:29:28.943050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943026    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume\") pod \"249b3e9b-4ba7-4757-b1c0-132718957d2f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") "
Sep 10 11:29:28.943177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943056    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57jcp\" (UniqueName: \"kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp\") pod \"249b3e9b-4ba7-4757-b1c0-132718957d2f\" (UID: \"249b3e9b-4ba7-4757-b1c0-132718957d2f\") "
Sep 10 11:29:28.943177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943114    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-854n7\" (UniqueName: \"kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7\") pod \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") "
Sep 10 11:29:28.943423 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943403    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume\") pod \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\" (UID: \"c0f12ec3-50d9-4603-94b4-43c94b7a5d06\") "
Sep 10 11:29:28.943423 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943408    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "249b3e9b-4ba7-4757-b1c0-132718957d2f" (UID: "249b3e9b-4ba7-4757-b1c0-132718957d2f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:29:28.943688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943666    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0f12ec3-50d9-4603-94b4-43c94b7a5d06" (UID: "c0f12ec3-50d9-4603-94b4-43c94b7a5d06"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:29:28.943788 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.943723    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/249b3e9b-4ba7-4757-b1c0-132718957d2f-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:28.944766 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.944747    2585 scope.go:117] "RemoveContainer" containerID="e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"
Sep 10 11:29:28.945204 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:29:28.945179    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29\": container with ID starting with e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29 not found: ID does not exist" containerID="e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"
Sep 10 11:29:28.945285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.945215    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29"} err="failed to get container status \"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29\": rpc error: code = NotFound desc = could not find container \"e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29\": container with ID starting with e3a694d1ad5e94b0f675d6cf6c524af656fa7bdb39879bab1d91182c4e034e29 not found: ID does not exist"
Sep 10 11:29:28.945285 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.945256    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp" (OuterVolumeSpecName: "kube-api-access-57jcp") pod "249b3e9b-4ba7-4757-b1c0-132718957d2f" (UID: "249b3e9b-4ba7-4757-b1c0-132718957d2f"). InnerVolumeSpecName "kube-api-access-57jcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:29:28.945384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.945295    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7" (OuterVolumeSpecName: "kube-api-access-854n7") pod "c0f12ec3-50d9-4603-94b4-43c94b7a5d06" (UID: "c0f12ec3-50d9-4603-94b4-43c94b7a5d06"). InnerVolumeSpecName "kube-api-access-854n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:29:28.993017 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:28.992999    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:29:29.044375 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.044265    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-854n7\" (UniqueName: \"kubernetes.io/projected/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-kube-api-access-854n7\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.044375 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.044291    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f12ec3-50d9-4603-94b4-43c94b7a5d06-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.044375 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.044321    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57jcp\" (UniqueName: \"kubernetes.io/projected/249b3e9b-4ba7-4757-b1c0-132718957d2f-kube-api-access-57jcp\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.124505 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.124479    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:29:29.144854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.144647    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume\") pod \"206574c2-4bd5-4e5e-8261-3061116d3740\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") "
Sep 10 11:29:29.144854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.144709    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h786t\" (UniqueName: \"kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t\") pod \"206574c2-4bd5-4e5e-8261-3061116d3740\" (UID: \"206574c2-4bd5-4e5e-8261-3061116d3740\") "
Sep 10 11:29:29.145150 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.145124    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "206574c2-4bd5-4e5e-8261-3061116d3740" (UID: "206574c2-4bd5-4e5e-8261-3061116d3740"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:29:29.145391 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.145372    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/206574c2-4bd5-4e5e-8261-3061116d3740-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.147334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.147287    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t" (OuterVolumeSpecName: "kube-api-access-h786t") pod "206574c2-4bd5-4e5e-8261-3061116d3740" (UID: "206574c2-4bd5-4e5e-8261-3061116d3740"). InnerVolumeSpecName "kube-api-access-h786t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:29:29.198656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.198631    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:29:29.204031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.203996    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0024ffd2-587cd6875c-jljnv"]
Sep 10 11:29:29.214618 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.214594    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:29:29.223825 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.223806    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0024ffd2-68d7f895c8-kxw5f"]
Sep 10 11:29:29.246358 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.246334    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume\") pod \"dac2e692-a221-4297-a471-7429b5383f37\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") "
Sep 10 11:29:29.246459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.246368    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7f9t\" (UniqueName: \"kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t\") pod \"dac2e692-a221-4297-a471-7429b5383f37\" (UID: \"dac2e692-a221-4297-a471-7429b5383f37\") "
Sep 10 11:29:29.246610 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.246598    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h786t\" (UniqueName: \"kubernetes.io/projected/206574c2-4bd5-4e5e-8261-3061116d3740-kube-api-access-h786t\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.246695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.246672    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dac2e692-a221-4297-a471-7429b5383f37" (UID: "dac2e692-a221-4297-a471-7429b5383f37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:29:29.248411 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.248391    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t" (OuterVolumeSpecName: "kube-api-access-l7f9t") pod "dac2e692-a221-4297-a471-7429b5383f37" (UID: "dac2e692-a221-4297-a471-7429b5383f37"). InnerVolumeSpecName "kube-api-access-l7f9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:29:29.347730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.347656    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dac2e692-a221-4297-a471-7429b5383f37-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.347730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.347681    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7f9t\" (UniqueName: \"kubernetes.io/projected/dac2e692-a221-4297-a471-7429b5383f37-kube-api-access-l7f9t\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:29.663756 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.663725    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="249b3e9b-4ba7-4757-b1c0-132718957d2f" path="/var/lib/kubelet/pods/249b3e9b-4ba7-4757-b1c0-132718957d2f/volumes"
Sep 10 11:29:29.664095 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.664074    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" path="/var/lib/kubelet/pods/c0f12ec3-50d9-4603-94b4-43c94b7a5d06/volumes"
Sep 10 11:29:29.887534 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.887505    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"
Sep 10 11:29:29.887693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.887532    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2" event={"ID":"206574c2-4bd5-4e5e-8261-3061116d3740","Type":"ContainerDied","Data":"db2e64f0395fa46b1576573685fe647842275ad9b53f019d731c86999b97824b"}
Sep 10 11:29:29.887693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.887601    2585 scope.go:117] "RemoveContainer" containerID="906106b935b96380dac7e72f4eca7fea8cf1f772c18ae0eba97723f3d80eff32"
Sep 10 11:29:29.888898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.888842    2585 generic.go:356] "Generic (PLEG): container finished" podID="dac2e692-a221-4297-a471-7429b5383f37" containerID="a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8" exitCode=0
Sep 10 11:29:29.888898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.888881    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"
Sep 10 11:29:29.889014 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.888906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" event={"ID":"dac2e692-a221-4297-a471-7429b5383f37","Type":"ContainerDied","Data":"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"}
Sep 10 11:29:29.889014 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.888946    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h" event={"ID":"dac2e692-a221-4297-a471-7429b5383f37","Type":"ContainerDied","Data":"a1784116d470dd60d561043d542a3951418bc3c78631c21d29c46a7b324932a9"}
Sep 10 11:29:29.896231 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.896212    2585 scope.go:117] "RemoveContainer" containerID="a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"
Sep 10 11:29:29.903701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.903683    2585 scope.go:117] "RemoveContainer" containerID="a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"
Sep 10 11:29:29.903971 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:29:29.903949    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8\": container with ID starting with a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8 not found: ID does not exist" containerID="a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"
Sep 10 11:29:29.904024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.903979    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8"} err="failed to get container status \"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8\": rpc error: code = NotFound desc = could not find container \"a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8\": container with ID starting with a8c040c4c4463eaac73c639142ff779d68159fc86a198cddc5405916e79470c8 not found: ID does not exist"
Sep 10 11:29:29.911537 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.911509    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:29:29.922906 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.922856    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8de26b-dcf4f46d6-qmq7h"]
Sep 10 11:29:29.933593 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.933569    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:29:29.937803 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:29.937785    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8de26b-97dd787f9-cklz2"]
Sep 10 11:29:31.538410 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.538372    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" podUID="64e2a07a-1996-4701-8d87-35f61028ce61" containerName="main" containerID="cri-o://99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e" gracePeriod=45
Sep 10 11:29:31.663540 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.663513    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="206574c2-4bd5-4e5e-8261-3061116d3740" path="/var/lib/kubelet/pods/206574c2-4bd5-4e5e-8261-3061116d3740/volumes"
Sep 10 11:29:31.663872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.663860    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dac2e692-a221-4297-a471-7429b5383f37" path="/var/lib/kubelet/pods/dac2e692-a221-4297-a471-7429b5383f37/volumes"
Sep 10 11:29:31.791760 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.791706    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:29:31.871327 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871280    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871344    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871394    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871419    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvvs7\" (UniqueName: \"kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871506    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871523    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:29:31.871801 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871531    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home\") pod \"64e2a07a-1996-4701-8d87-35f61028ce61\" (UID: \"64e2a07a-1996-4701-8d87-35f61028ce61\") "
Sep 10 11:29:31.871801 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871737    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache" (OuterVolumeSpecName: "model-cache") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:29:31.871801 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871750    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home" (OuterVolumeSpecName: "home") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:29:31.871904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871886    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:31.871904 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871900    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:31.871973 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.871910    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:31.873471 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.873453    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm" (OuterVolumeSpecName: "dshm") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:29:31.873867 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.873847    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:29:31.873919 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.873877    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7" (OuterVolumeSpecName: "kube-api-access-cvvs7") pod "64e2a07a-1996-4701-8d87-35f61028ce61" (UID: "64e2a07a-1996-4701-8d87-35f61028ce61"). InnerVolumeSpecName "kube-api-access-cvvs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:29:31.905046 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.905017    2585 generic.go:356] "Generic (PLEG): container finished" podID="64e2a07a-1996-4701-8d87-35f61028ce61" containerID="99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e" exitCode=0
Sep 10 11:29:31.905136 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.905067    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"
Sep 10 11:29:31.905136 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.905101    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" event={"ID":"64e2a07a-1996-4701-8d87-35f61028ce61","Type":"ContainerDied","Data":"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"}
Sep 10 11:29:31.905238 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.905143    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd" event={"ID":"64e2a07a-1996-4701-8d87-35f61028ce61","Type":"ContainerDied","Data":"b099fa9de5f1b4aa1a5fc76be9a816b17aa0bc9cc5a202dff3d3d659ecc1bb65"}
Sep 10 11:29:31.905238 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.905160    2585 scope.go:117] "RemoveContainer" containerID="99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"
Sep 10 11:29:31.914704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.914682    2585 scope.go:117] "RemoveContainer" containerID="99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"
Sep 10 11:29:31.915013 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:29:31.914994    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e\": container with ID starting with 99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e not found: ID does not exist" containerID="99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"
Sep 10 11:29:31.915091 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.915018    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e"} err="failed to get container status \"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e\": rpc error: code = NotFound desc = could not find container \"99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e\": container with ID starting with 99fe030aab0d562e859a901767bdf8cdb045de39819c59c4323b5959494ce31e not found: ID does not exist"
Sep 10 11:29:31.933489 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.933467    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:29:31.936901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.936881    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-27987e85-kserve-7f7699fd9b-jzshd"]
Sep 10 11:29:31.972550 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.972530    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64e2a07a-1996-4701-8d87-35f61028ce61-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:31.972550 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.972548    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/64e2a07a-1996-4701-8d87-35f61028ce61-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:31.972650 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:31.972557    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvvs7\" (UniqueName: \"kubernetes.io/projected/64e2a07a-1996-4701-8d87-35f61028ce61-kube-api-access-cvvs7\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:29:33.662782 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:33.662743    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64e2a07a-1996-4701-8d87-35f61028ce61" path="/var/lib/kubelet/pods/64e2a07a-1996-4701-8d87-35f61028ce61/volumes"
Sep 10 11:29:36.052693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:36.052660    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:29:43.247328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247283    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247658    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" containerName="payload-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247670    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" containerName="payload-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247682    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac2e692-a221-4297-a471-7429b5383f37" containerName="payload-pre-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247689    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac2e692-a221-4297-a471-7429b5383f37" containerName="payload-pre-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247703    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="249b3e9b-4ba7-4757-b1c0-132718957d2f" containerName="payload-pre-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247710    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="249b3e9b-4ba7-4757-b1c0-132718957d2f" containerName="payload-pre-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247722    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="206574c2-4bd5-4e5e-8261-3061116d3740" containerName="payload-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247727    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="206574c2-4bd5-4e5e-8261-3061116d3740" containerName="payload-processing"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247737    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64e2a07a-1996-4701-8d87-35f61028ce61" containerName="main"
Sep 10 11:29:43.247773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247742    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="64e2a07a-1996-4701-8d87-35f61028ce61" containerName="main"
Sep 10 11:29:43.248126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247797    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f12ec3-50d9-4603-94b4-43c94b7a5d06" containerName="payload-processing"
Sep 10 11:29:43.248126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247804    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="249b3e9b-4ba7-4757-b1c0-132718957d2f" containerName="payload-pre-processing"
Sep 10 11:29:43.248126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247811    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac2e692-a221-4297-a471-7429b5383f37" containerName="payload-pre-processing"
Sep 10 11:29:43.248126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247820    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="64e2a07a-1996-4701-8d87-35f61028ce61" containerName="main"
Sep 10 11:29:43.248126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.247826    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="206574c2-4bd5-4e5e-8261-3061116d3740" containerName="payload-processing"
Sep 10 11:29:43.252249 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.252229    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.254926 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.254906    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-f39de1\""
Sep 10 11:29:43.255062 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.255042    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-f39de1-dockercfg-btqqn\""
Sep 10 11:29:43.260602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.260584    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:29:43.271793 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.271769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nbkd\" (UniqueName: \"kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.271901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.271833    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.372960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.372932    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.373116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.372986    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nbkd\" (UniqueName: \"kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.373660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.373639    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.384917 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.384895    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nbkd\" (UniqueName: \"kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd\") pod \"payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") " pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.563611 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.563515    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.841942 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.841916    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:29:43.844749 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:29:43.844721    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959661ba_3500_4ed5_8e34_c015edef9c27.slice/crio-ff6282f6f5129cd290748c63a8a8d59025659eb642302c3a7bca37b087fe2383 WatchSource:0}: Error finding container ff6282f6f5129cd290748c63a8a8d59025659eb642302c3a7bca37b087fe2383: Status 404 returned error can't find the container with id ff6282f6f5129cd290748c63a8a8d59025659eb642302c3a7bca37b087fe2383
Sep 10 11:29:43.965642 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.965600    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" event={"ID":"959661ba-3500-4ed5-8e34-c015edef9c27","Type":"ContainerStarted","Data":"8b1583601069201ac517a0d3bcb1aba383464653be94e58df59e179c230b246d"}
Sep 10 11:29:43.965811 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.965662    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" event={"ID":"959661ba-3500-4ed5-8e34-c015edef9c27","Type":"ContainerStarted","Data":"ff6282f6f5129cd290748c63a8a8d59025659eb642302c3a7bca37b087fe2383"}
Sep 10 11:29:43.965811 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.965685    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:29:43.984271 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:43.984227    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" podStartSLOduration=0.984212324 podStartE2EDuration="984.212324ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:29:43.983190538 +0000 UTC m=+918.787257886" watchObservedRunningTime="2026-09-10 11:29:43.984212324 +0000 UTC m=+918.788279671"
Sep 10 11:29:50.712704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:50.712666    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:29:54.973249 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:29:54.973214    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:30:00.163527 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.163491    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"]
Sep 10 11:30:00.166895 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.166878    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"
Sep 10 11:30:00.172206 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.172186    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:30:00.176052 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.176037    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"
Sep 10 11:30:00.186873 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.186846    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"]
Sep 10 11:30:00.189273 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.189248    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"]
Sep 10 11:30:00.193444 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.193425    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"
Sep 10 11:30:00.201558 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.201513    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"]
Sep 10 11:30:00.215504 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.215476    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"
Sep 10 11:30:00.463875 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.463851    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"]
Sep 10 11:30:00.466542 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:30:00.466508    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b19c43_957d_4dc0_8b03_20915a0039bd.slice/crio-29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe WatchSource:0}: Error finding container 29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe: Status 404 returned error can't find the container with id 29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe
Sep 10 11:30:00.498754 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:00.498513    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"]
Sep 10 11:30:00.500444 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:30:00.500420    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734c1900_e2e4_40ef_80dc_5c0aaf01f8a9.slice/crio-61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c WatchSource:0}: Error finding container 61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c: Status 404 returned error can't find the container with id 61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c
Sep 10 11:30:01.054492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.054405    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p" event={"ID":"734c1900-e2e4-40ef-80dc-5c0aaf01f8a9","Type":"ContainerStarted","Data":"1a00538e052b586dab397fae5a2db5fe2ff166180cc74a7311ced2e1149b08a3"}
Sep 10 11:30:01.054492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.054461    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p" event={"ID":"734c1900-e2e4-40ef-80dc-5c0aaf01f8a9","Type":"ContainerStarted","Data":"61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c"}
Sep 10 11:30:01.056220 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.056190    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5" event={"ID":"62b19c43-957d-4dc0-8b03-20915a0039bd","Type":"ContainerStarted","Data":"b356cb8f72b5b120e4b687de0e2236aa21c018e37f284d563c00a06dd000071f"}
Sep 10 11:30:01.056382 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.056236    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5" event={"ID":"62b19c43-957d-4dc0-8b03-20915a0039bd","Type":"ContainerStarted","Data":"29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe"}
Sep 10 11:30:01.072281 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.072238    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p" podStartSLOduration=1.072223597 podStartE2EDuration="1.072223597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:30:01.070041737 +0000 UTC m=+935.874109085" watchObservedRunningTime="2026-09-10 11:30:01.072223597 +0000 UTC m=+935.876290944"
Sep 10 11:30:01.092130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:01.092082    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5" podStartSLOduration=1.092070809 podStartE2EDuration="1.092070809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:30:01.091353532 +0000 UTC m=+935.895420880" watchObservedRunningTime="2026-09-10 11:30:01.092070809 +0000 UTC m=+935.896138216"
Sep 10 11:30:02.062704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:02.062668    2585 generic.go:356] "Generic (PLEG): container finished" podID="62b19c43-957d-4dc0-8b03-20915a0039bd" containerID="b356cb8f72b5b120e4b687de0e2236aa21c018e37f284d563c00a06dd000071f" exitCode=0
Sep 10 11:30:02.063154 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:02.062753    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5" event={"ID":"62b19c43-957d-4dc0-8b03-20915a0039bd","Type":"ContainerDied","Data":"b356cb8f72b5b120e4b687de0e2236aa21c018e37f284d563c00a06dd000071f"}
Sep 10 11:30:02.064260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:02.064238    2585 generic.go:356] "Generic (PLEG): container finished" podID="734c1900-e2e4-40ef-80dc-5c0aaf01f8a9" containerID="1a00538e052b586dab397fae5a2db5fe2ff166180cc74a7311ced2e1149b08a3" exitCode=0
Sep 10 11:30:02.064382 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:02.064282    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p" event={"ID":"734c1900-e2e4-40ef-80dc-5c0aaf01f8a9","Type":"ContainerDied","Data":"1a00538e052b586dab397fae5a2db5fe2ff166180cc74a7311ced2e1149b08a3"}
Sep 10 11:30:03.202862 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:03.202836    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"
Sep 10 11:30:03.241962 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:03.241940    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"
Sep 10 11:30:03.562681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:03.562640    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:30:04.075745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.075660    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"
Sep 10 11:30:04.075745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.075683    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p" event={"ID":"734c1900-e2e4-40ef-80dc-5c0aaf01f8a9","Type":"ContainerDied","Data":"61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c"}
Sep 10 11:30:04.075745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.075729    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61ffe7e2931882f8085ed8dc5e286b51614d52262ecb1c0b4cd7abc4c9b4338c"
Sep 10 11:30:04.077004 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.076984    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5"
Sep 10 11:30:04.077119 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.076985    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817330-b9dc5" event={"ID":"62b19c43-957d-4dc0-8b03-20915a0039bd","Type":"ContainerDied","Data":"29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe"}
Sep 10 11:30:04.077160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.077128    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29c2aeb64ae7e6723d3b08c7f57a539ea252ed906c2b2bd49aa307a92d9827fe"
Sep 10 11:30:04.378387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.378291    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:30:04.378905 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.378885    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="734c1900-e2e4-40ef-80dc-5c0aaf01f8a9" containerName="cleanup"
Sep 10 11:30:04.378984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.378907    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="734c1900-e2e4-40ef-80dc-5c0aaf01f8a9" containerName="cleanup"
Sep 10 11:30:04.378984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.378923    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62b19c43-957d-4dc0-8b03-20915a0039bd" containerName="cleanup"
Sep 10 11:30:04.378984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.378931    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="62b19c43-957d-4dc0-8b03-20915a0039bd" containerName="cleanup"
Sep 10 11:30:04.379143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.379024    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="734c1900-e2e4-40ef-80dc-5c0aaf01f8a9" containerName="cleanup"
Sep 10 11:30:04.379143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.379039    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="62b19c43-957d-4dc0-8b03-20915a0039bd" containerName="cleanup"
Sep 10 11:30:04.381984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.381964    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.384389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.384366    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4c9990\"/\"default-dockercfg-82qxh\""
Sep 10 11:30:04.384499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.384374    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4c9990\"/\"openshift-service-ca.crt\""
Sep 10 11:30:04.384499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.384409    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4c9990\"/\"kube-root-ca.crt\""
Sep 10 11:30:04.384622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.384544    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4c9990\"/\"test-model-4c9990-kserve-self-signed-certs\""
Sep 10 11:30:04.391602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.391581    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:30:04.460688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460657    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.460856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460708    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.460856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460776    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbmc5\" (UniqueName: \"kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.460856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460798    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.460856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460815    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.461036 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.460917    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561379 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561564 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561403    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561564 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561438    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbmc5\" (UniqueName: \"kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561564 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561462    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561564 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561564 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561519    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561852 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561776    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561852 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561815    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.561852 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.561837    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.563796 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.563774    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.564026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.564010    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.570197 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.570173    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbmc5\" (UniqueName: \"kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5\") pod \"test-model-4c9990-kserve-86765bf7b5-vrgb7\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") " pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.691845 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.691814    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:04.968414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:04.968390    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:30:04.971085 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:30:04.971059    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ef3b694_228c_4d98_8abb_fe5e5e13495a.slice/crio-6f08c303a8eec276191bf5bc99f6f918bb0ee1c15483b836c853857c5ef39303 WatchSource:0}: Error finding container 6f08c303a8eec276191bf5bc99f6f918bb0ee1c15483b836c853857c5ef39303: Status 404 returned error can't find the container with id 6f08c303a8eec276191bf5bc99f6f918bb0ee1c15483b836c853857c5ef39303
Sep 10 11:30:05.083509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:05.083466    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" event={"ID":"9ef3b694-228c-4d98-8abb-fe5e5e13495a","Type":"ContainerStarted","Data":"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"}
Sep 10 11:30:05.083692 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:05.083518    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" event={"ID":"9ef3b694-228c-4d98-8abb-fe5e5e13495a","Type":"ContainerStarted","Data":"6f08c303a8eec276191bf5bc99f6f918bb0ee1c15483b836c853857c5ef39303"}
Sep 10 11:30:05.102384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:05.102340    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" podStartSLOduration=1.102294505 podStartE2EDuration="1.102294505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:30:05.100491684 +0000 UTC m=+939.904559033" watchObservedRunningTime="2026-09-10 11:30:05.102294505 +0000 UTC m=+939.906361855"
Sep 10 11:30:14.692459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:14.692427    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:14.692928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:14.692482    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:14.708829 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:14.708797    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:15.143361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:15.143266    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:30:17.544938 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:17.544903    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:30:36.476909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:36.476874    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:30:38.038259 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.038227    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"]
Sep 10 11:30:38.040843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.040817    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-f39de1-29817330-d268p"]
Sep 10 11:30:38.076387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.076357    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:30:38.076605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.076558    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" podUID="959661ba-3500-4ed5-8e34-c015edef9c27" containerName="payload-processing" containerID="cri-o://8b1583601069201ac517a0d3bcb1aba383464653be94e58df59e179c230b246d" gracePeriod=30
Sep 10 11:30:38.248979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.248948    2585 generic.go:356] "Generic (PLEG): container finished" podID="959661ba-3500-4ed5-8e34-c015edef9c27" containerID="8b1583601069201ac517a0d3bcb1aba383464653be94e58df59e179c230b246d" exitCode=0
Sep 10 11:30:38.249124 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.248979    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" event={"ID":"959661ba-3500-4ed5-8e34-c015edef9c27","Type":"ContainerDied","Data":"8b1583601069201ac517a0d3bcb1aba383464653be94e58df59e179c230b246d"}
Sep 10 11:30:38.329919 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.329894    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:30:38.368485 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.368458    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nbkd\" (UniqueName: \"kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd\") pod \"959661ba-3500-4ed5-8e34-c015edef9c27\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") "
Sep 10 11:30:38.368639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.368511    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume\") pod \"959661ba-3500-4ed5-8e34-c015edef9c27\" (UID: \"959661ba-3500-4ed5-8e34-c015edef9c27\") "
Sep 10 11:30:38.368912 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.368886    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "959661ba-3500-4ed5-8e34-c015edef9c27" (UID: "959661ba-3500-4ed5-8e34-c015edef9c27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:30:38.370556 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.370532    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd" (OuterVolumeSpecName: "kube-api-access-4nbkd") pod "959661ba-3500-4ed5-8e34-c015edef9c27" (UID: "959661ba-3500-4ed5-8e34-c015edef9c27"). InnerVolumeSpecName "kube-api-access-4nbkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:30:38.470207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.470176    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/959661ba-3500-4ed5-8e34-c015edef9c27-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:30:38.470207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:38.470202    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nbkd\" (UniqueName: \"kubernetes.io/projected/959661ba-3500-4ed5-8e34-c015edef9c27-kube-api-access-4nbkd\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:30:39.255884 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.255853    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"
Sep 10 11:30:39.255884 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.255874    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq" event={"ID":"959661ba-3500-4ed5-8e34-c015edef9c27","Type":"ContainerDied","Data":"ff6282f6f5129cd290748c63a8a8d59025659eb642302c3a7bca37b087fe2383"}
Sep 10 11:30:39.256418 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.255933    2585 scope.go:117] "RemoveContainer" containerID="8b1583601069201ac517a0d3bcb1aba383464653be94e58df59e179c230b246d"
Sep 10 11:30:39.277338 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.277271    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:30:39.284835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.284810    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-f39de1-78846c78f-tq9dq"]
Sep 10 11:30:39.668256 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.664761    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="734c1900-e2e4-40ef-80dc-5c0aaf01f8a9" path="/var/lib/kubelet/pods/734c1900-e2e4-40ef-80dc-5c0aaf01f8a9/volumes"
Sep 10 11:30:39.668256 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:39.665291    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="959661ba-3500-4ed5-8e34-c015edef9c27" path="/var/lib/kubelet/pods/959661ba-3500-4ed5-8e34-c015edef9c27/volumes"
Sep 10 11:30:50.590100 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:30:50.590023    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:02.176855 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:02.176818    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:11.789195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.789161    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:11.789598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.789570    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="959661ba-3500-4ed5-8e34-c015edef9c27" containerName="payload-processing"
Sep 10 11:31:11.789598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.789583    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="959661ba-3500-4ed5-8e34-c015edef9c27" containerName="payload-processing"
Sep 10 11:31:11.789670 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.789662    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="959661ba-3500-4ed5-8e34-c015edef9c27" containerName="payload-processing"
Sep 10 11:31:11.792500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.792486    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.794941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.794909    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-395207\"/\"openshift-service-ca.crt\""
Sep 10 11:31:11.794941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.794911    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-395207\"/\"kube-root-ca.crt\""
Sep 10 11:31:11.795438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.795422    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-395207\"/\"default-dockercfg-55spc\""
Sep 10 11:31:11.795538 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.795454    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-395207\"/\"noten-model-395207-kserve-self-signed-certs\""
Sep 10 11:31:11.808138 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.808113    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:11.870659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870621    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.870659 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870660    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.870922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870736    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.870922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870835    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.870922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870861    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.870922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.870884    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kmvm\" (UniqueName: \"kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972209    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972238    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972287    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972336    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972361    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmvm\" (UniqueName: \"kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972663 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972605    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972714 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972696    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.972762 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.972735    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.974600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.974580    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.974895 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.974875    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:11.983289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:11.983266    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kmvm\" (UniqueName: \"kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm\") pod \"noten-model-395207-kserve-8567fdff7f-f6d8q\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") " pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:12.101975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:12.101881    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:12.102674 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:12.102644    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:12.378974 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:12.378946    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:12.381238 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:31:12.381207    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod973b0e69_0301_46ac_9981_bd422f8d5bba.slice/crio-0b2adbb48f7e03a4c4a33a53fa171443d1601c660ae878ac47b81ade62ca981e WatchSource:0}: Error finding container 0b2adbb48f7e03a4c4a33a53fa171443d1601c660ae878ac47b81ade62ca981e: Status 404 returned error can't find the container with id 0b2adbb48f7e03a4c4a33a53fa171443d1601c660ae878ac47b81ade62ca981e
Sep 10 11:31:12.430441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:12.430411    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" event={"ID":"973b0e69-0301-46ac-9981-bd422f8d5bba","Type":"ContainerStarted","Data":"0b2adbb48f7e03a4c4a33a53fa171443d1601c660ae878ac47b81ade62ca981e"}
Sep 10 11:31:13.437560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:13.437525    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" event={"ID":"973b0e69-0301-46ac-9981-bd422f8d5bba","Type":"ContainerStarted","Data":"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"}
Sep 10 11:31:13.456202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:13.456149    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" podStartSLOduration=2.456132309 podStartE2EDuration="2.456132309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:31:13.455546107 +0000 UTC m=+1008.259613468" watchObservedRunningTime="2026-09-10 11:31:13.456132309 +0000 UTC m=+1008.260199919"
Sep 10 11:31:15.132670 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:15.132639    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:16.686999 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.686962    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:16.690212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.690195    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.697802 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.697774    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5cf411f1\""
Sep 10 11:31:16.700197 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.700176    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5cf411f1-dockercfg-ngd5n\""
Sep 10 11:31:16.715542 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.715517    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.715697 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.715665    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8cjz\" (UniqueName: \"kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.718203 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.718180    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:16.816956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.816925    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8cjz\" (UniqueName: \"kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.817125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.816988    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.817954 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.817906    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:16.825907 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:16.825874    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8cjz\" (UniqueName: \"kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz\") pod \"payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:17.001357 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.001234    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:17.283877 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.283842    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:17.285721 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:31:17.285689    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68104a59_9a57_42ea_a0ef_22581b8c3a42.slice/crio-654e6a6d75959a342aefd6019a38953cc2424fc5551ce8c41392e62968601dca WatchSource:0}: Error finding container 654e6a6d75959a342aefd6019a38953cc2424fc5551ce8c41392e62968601dca: Status 404 returned error can't find the container with id 654e6a6d75959a342aefd6019a38953cc2424fc5551ce8c41392e62968601dca
Sep 10 11:31:17.460218 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.460183    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" event={"ID":"68104a59-9a57-42ea-a0ef-22581b8c3a42","Type":"ContainerStarted","Data":"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"}
Sep 10 11:31:17.460445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.460245    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" event={"ID":"68104a59-9a57-42ea-a0ef-22581b8c3a42","Type":"ContainerStarted","Data":"654e6a6d75959a342aefd6019a38953cc2424fc5551ce8c41392e62968601dca"}
Sep 10 11:31:17.460445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.460288    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:17.484539 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:17.484492    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" podStartSLOduration=1.484477482 podStartE2EDuration="1.484477482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:31:17.483194891 +0000 UTC m=+1012.287262250" watchObservedRunningTime="2026-09-10 11:31:17.484477482 +0000 UTC m=+1012.288544829"
Sep 10 11:31:21.155637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.155599    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:31:21.260459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.260420    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"]
Sep 10 11:31:21.264016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.263995    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:21.266488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.266469    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:31:21.266578 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.266494    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:31:21.272948 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.272923    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"]
Sep 10 11:31:21.357350 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.357287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn\" (UID: \"50b8e702-bd72-429c-a8cd-3708a10261b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:21.458122 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.458095    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn\" (UID: \"50b8e702-bd72-429c-a8cd-3708a10261b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:21.458729 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.458708    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn\" (UID: \"50b8e702-bd72-429c-a8cd-3708a10261b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:21.575706 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.575678    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:21.866992 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:21.866877    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"]
Sep 10 11:31:21.869393 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:31:21.869367    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b8e702_bd72_429c_a8cd_3708a10261b6.slice/crio-5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b WatchSource:0}: Error finding container 5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b: Status 404 returned error can't find the container with id 5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b
Sep 10 11:31:22.104133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:22.104103    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:22.485849 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:22.485818    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn" event={"ID":"50b8e702-bd72-429c-a8cd-3708a10261b6","Type":"ContainerStarted","Data":"dfb7bedc186c60fcf6291b0ba7adb7fe1dc5deb029efadeb674cd82eb97a1a3d"}
Sep 10 11:31:22.486252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:22.485870    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn" event={"ID":"50b8e702-bd72-429c-a8cd-3708a10261b6","Type":"ContainerStarted","Data":"5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b"}
Sep 10 11:31:22.508244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:22.508191    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn" podStartSLOduration=1.5081741929999999 podStartE2EDuration="1.508174193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:31:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:31:22.507245092 +0000 UTC m=+1017.311312441" watchObservedRunningTime="2026-09-10 11:31:22.508174193 +0000 UTC m=+1017.312241541"
Sep 10 11:31:23.492507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:23.492472    2585 generic.go:356] "Generic (PLEG): container finished" podID="50b8e702-bd72-429c-a8cd-3708a10261b6" containerID="dfb7bedc186c60fcf6291b0ba7adb7fe1dc5deb029efadeb674cd82eb97a1a3d" exitCode=0
Sep 10 11:31:23.492865 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:23.492560    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn" event={"ID":"50b8e702-bd72-429c-a8cd-3708a10261b6","Type":"ContainerDied","Data":"dfb7bedc186c60fcf6291b0ba7adb7fe1dc5deb029efadeb674cd82eb97a1a3d"}
Sep 10 11:31:24.629865 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:24.629841    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:24.683577 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:24.683543    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca\") pod \"50b8e702-bd72-429c-a8cd-3708a10261b6\" (UID: \"50b8e702-bd72-429c-a8cd-3708a10261b6\") "
Sep 10 11:31:24.683872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:24.683849    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50b8e702-bd72-429c-a8cd-3708a10261b6" (UID: "50b8e702-bd72-429c-a8cd-3708a10261b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:24.784130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:24.784066    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b8e702-bd72-429c-a8cd-3708a10261b6-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:25.482090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:25.482057    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:25.504557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:25.504528    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn" event={"ID":"50b8e702-bd72-429c-a8cd-3708a10261b6","Type":"ContainerDied","Data":"5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b"}
Sep 10 11:31:25.504557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:25.504547    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"
Sep 10 11:31:25.504737 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:25.504572    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5552360a1b275c24fbd1cf94769c51730df5ba260216ea724170e9945eb92e8b"
Sep 10 11:31:28.467402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:28.467365    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:30.467045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.466990    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" podUID="973b0e69-0301-46ac-9981-bd422f8d5bba" containerName="main" containerID="cri-o://2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf" gracePeriod=45
Sep 10 11:31:30.716151 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.716122    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:30.839214 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839144    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839214 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839180    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kmvm\" (UniqueName: \"kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839258    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839329    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839366    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839445 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839400    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm\") pod \"973b0e69-0301-46ac-9981-bd422f8d5bba\" (UID: \"973b0e69-0301-46ac-9981-bd422f8d5bba\") "
Sep 10 11:31:30.839664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839561    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache" (OuterVolumeSpecName: "model-cache") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:30.839664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839579    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:30.839755 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839653    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home" (OuterVolumeSpecName: "home") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:30.839755 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839740    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:30.839824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839761    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:30.839824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.839775    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:30.841639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.841607    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm" (OuterVolumeSpecName: "dshm") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:30.841848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.841829    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm" (OuterVolumeSpecName: "kube-api-access-2kmvm") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "kube-api-access-2kmvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:30.841848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.841829    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "973b0e69-0301-46ac-9981-bd422f8d5bba" (UID: "973b0e69-0301-46ac-9981-bd422f8d5bba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:31:30.941135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.941106    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/973b0e69-0301-46ac-9981-bd422f8d5bba-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:30.941135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.941130    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/973b0e69-0301-46ac-9981-bd422f8d5bba-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:30.941135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:30.941140    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kmvm\" (UniqueName: \"kubernetes.io/projected/973b0e69-0301-46ac-9981-bd422f8d5bba-kube-api-access-2kmvm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:31.541000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.540968    2585 generic.go:356] "Generic (PLEG): container finished" podID="973b0e69-0301-46ac-9981-bd422f8d5bba" containerID="2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf" exitCode=0
Sep 10 11:31:31.541437 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.541008    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"
Sep 10 11:31:31.541437 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.541034    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" event={"ID":"973b0e69-0301-46ac-9981-bd422f8d5bba","Type":"ContainerDied","Data":"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"}
Sep 10 11:31:31.541437 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.541065    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q" event={"ID":"973b0e69-0301-46ac-9981-bd422f8d5bba","Type":"ContainerDied","Data":"0b2adbb48f7e03a4c4a33a53fa171443d1601c660ae878ac47b81ade62ca981e"}
Sep 10 11:31:31.541437 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.541082    2585 scope.go:117] "RemoveContainer" containerID="2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"
Sep 10 11:31:31.550515 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.550495    2585 scope.go:117] "RemoveContainer" containerID="2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"
Sep 10 11:31:31.550782 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:31.550762    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf\": container with ID starting with 2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf not found: ID does not exist" containerID="2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"
Sep 10 11:31:31.550853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.550789    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf"} err="failed to get container status \"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf\": rpc error: code = NotFound desc = could not find container \"2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf\": container with ID starting with 2a9cc9c3dc2da168e0c37d7b0f07f4b8dbadbd131713551e7f9fe6575f700fbf not found: ID does not exist"
Sep 10 11:31:31.572127 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.572103    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:31.584228 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.584208    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-395207/noten-model-395207-kserve-8567fdff7f-f6d8q"]
Sep 10 11:31:31.663616 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:31.663587    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="973b0e69-0301-46ac-9981-bd422f8d5bba" path="/var/lib/kubelet/pods/973b0e69-0301-46ac-9981-bd422f8d5bba/volumes"
Sep 10 11:31:32.106673 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.106641    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:31:32.107103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.107051    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" podUID="4e60f345-19a0-466e-95a9-2c09011aef8f" containerName="payload-processing" containerID="cri-o://70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623" gracePeriod=30
Sep 10 11:31:32.348927 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.348893    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"]
Sep 10 11:31:32.352401 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.352374    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8355df-v6mqn"]
Sep 10 11:31:32.371643 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.371580    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:31:32.456435 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.456401    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fklv\" (UniqueName: \"kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv\") pod \"4e60f345-19a0-466e-95a9-2c09011aef8f\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") "
Sep 10 11:31:32.456599 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.456457    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume\") pod \"4e60f345-19a0-466e-95a9-2c09011aef8f\" (UID: \"4e60f345-19a0-466e-95a9-2c09011aef8f\") "
Sep 10 11:31:32.456941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.456912    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e60f345-19a0-466e-95a9-2c09011aef8f" (UID: "4e60f345-19a0-466e-95a9-2c09011aef8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:32.458662 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.458631    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv" (OuterVolumeSpecName: "kube-api-access-2fklv") pod "4e60f345-19a0-466e-95a9-2c09011aef8f" (UID: "4e60f345-19a0-466e-95a9-2c09011aef8f"). InnerVolumeSpecName "kube-api-access-2fklv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:32.548009 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.547974    2585 generic.go:356] "Generic (PLEG): container finished" podID="4e60f345-19a0-466e-95a9-2c09011aef8f" containerID="70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623" exitCode=0
Sep 10 11:31:32.548492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.548023    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"
Sep 10 11:31:32.548492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.548032    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" event={"ID":"4e60f345-19a0-466e-95a9-2c09011aef8f","Type":"ContainerDied","Data":"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"}
Sep 10 11:31:32.548492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.548072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc" event={"ID":"4e60f345-19a0-466e-95a9-2c09011aef8f","Type":"ContainerDied","Data":"ec37898e4c14204ac8080124e9a762b83cb9be059f390233c9c9ff28032a3681"}
Sep 10 11:31:32.548492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.548098    2585 scope.go:117] "RemoveContainer" containerID="70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"
Sep 10 11:31:32.557419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.557394    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fklv\" (UniqueName: \"kubernetes.io/projected/4e60f345-19a0-466e-95a9-2c09011aef8f-kube-api-access-2fklv\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:32.557419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.557420    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e60f345-19a0-466e-95a9-2c09011aef8f-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:32.557697 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.557683    2585 scope.go:117] "RemoveContainer" containerID="70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"
Sep 10 11:31:32.557941 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:32.557921    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623\": container with ID starting with 70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623 not found: ID does not exist" containerID="70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"
Sep 10 11:31:32.557984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.557948    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623"} err="failed to get container status \"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623\": rpc error: code = NotFound desc = could not find container \"70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623\": container with ID starting with 70f730cb557f606d2dbc1c35957aa1ba9d6b7548fa34c94cccc8e29083fa8623 not found: ID does not exist"
Sep 10 11:31:32.573528 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.573500    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:31:32.578880 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:32.578858    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-8355df-6b7b5459dc-4rsdc"]
Sep 10 11:31:33.668783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:33.668745    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e60f345-19a0-466e-95a9-2c09011aef8f" path="/var/lib/kubelet/pods/4e60f345-19a0-466e-95a9-2c09011aef8f/volumes"
Sep 10 11:31:33.669152 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:33.669115    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50b8e702-bd72-429c-a8cd-3708a10261b6" path="/var/lib/kubelet/pods/50b8e702-bd72-429c-a8cd-3708a10261b6/volumes"
Sep 10 11:31:34.572187 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:34.572146    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:36.175153 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.175117    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" podUID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" containerName="main" containerID="cri-o://23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a" gracePeriod=45
Sep 10 11:31:36.424633 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.424610    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:31:36.494750 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494683    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbmc5\" (UniqueName: \"kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.494750 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494730    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.494956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494754    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.494956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494770    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.494956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494808    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.494956 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.494824    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs\") pod \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\" (UID: \"9ef3b694-228c-4d98-8abb-fe5e5e13495a\") "
Sep 10 11:31:36.495159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.495001    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:36.495159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.495015    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home" (OuterVolumeSpecName: "home") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:36.495159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.495083    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:36.495159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.495117    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:36.495159 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.495135    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:36.496950 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.496920    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:31:36.497059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.496964    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm" (OuterVolumeSpecName: "dshm") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:31:36.497059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.496985    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5" (OuterVolumeSpecName: "kube-api-access-lbmc5") pod "9ef3b694-228c-4d98-8abb-fe5e5e13495a" (UID: "9ef3b694-228c-4d98-8abb-fe5e5e13495a"). InnerVolumeSpecName "kube-api-access-lbmc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:36.570069 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.570044    2585 generic.go:356] "Generic (PLEG): container finished" podID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" containerID="23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a" exitCode=0
Sep 10 11:31:36.570217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.570087    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"
Sep 10 11:31:36.570217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.570081    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" event={"ID":"9ef3b694-228c-4d98-8abb-fe5e5e13495a","Type":"ContainerDied","Data":"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"}
Sep 10 11:31:36.570217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.570201    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7" event={"ID":"9ef3b694-228c-4d98-8abb-fe5e5e13495a","Type":"ContainerDied","Data":"6f08c303a8eec276191bf5bc99f6f918bb0ee1c15483b836c853857c5ef39303"}
Sep 10 11:31:36.570400 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.570221    2585 scope.go:117] "RemoveContainer" containerID="23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"
Sep 10 11:31:36.578733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.578717    2585 scope.go:117] "RemoveContainer" containerID="23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"
Sep 10 11:31:36.579010 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:36.578991    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a\": container with ID starting with 23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a not found: ID does not exist" containerID="23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"
Sep 10 11:31:36.579063 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.579018    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a"} err="failed to get container status \"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a\": rpc error: code = NotFound desc = could not find container \"23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a\": container with ID starting with 23cee6efbf825a4829883c528dd2cbb5424f53e841c331647dbff2f7cf9ba01a not found: ID does not exist"
Sep 10 11:31:36.590643 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.590617    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:31:36.595399 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.595362    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4c9990/test-model-4c9990-kserve-86765bf7b5-vrgb7"]
Sep 10 11:31:36.595664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.595649    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:36.595703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.595667    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ef3b694-228c-4d98-8abb-fe5e5e13495a-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:36.595703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.595676    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ef3b694-228c-4d98-8abb-fe5e5e13495a-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:36.595703 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:36.595684    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbmc5\" (UniqueName: \"kubernetes.io/projected/9ef3b694-228c-4d98-8abb-fe5e5e13495a-kube-api-access-lbmc5\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:37.663098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:37.663065    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" path="/var/lib/kubelet/pods/9ef3b694-228c-4d98-8abb-fe5e5e13495a/volumes"
Sep 10 11:31:39.787646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.787614    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"]
Sep 10 11:31:39.787997 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.787971    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" containerName="main"
Sep 10 11:31:39.787997 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.787982    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" containerName="main"
Sep 10 11:31:39.787997 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.787995    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b8e702-bd72-429c-a8cd-3708a10261b6" containerName="revoke-keys"
Sep 10 11:31:39.787997 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788000    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b8e702-bd72-429c-a8cd-3708a10261b6" containerName="revoke-keys"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788009    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e60f345-19a0-466e-95a9-2c09011aef8f" containerName="payload-processing"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788014    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e60f345-19a0-466e-95a9-2c09011aef8f" containerName="payload-processing"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788031    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="973b0e69-0301-46ac-9981-bd422f8d5bba" containerName="main"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788037    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="973b0e69-0301-46ac-9981-bd422f8d5bba" containerName="main"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788090    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="973b0e69-0301-46ac-9981-bd422f8d5bba" containerName="main"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788099    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e60f345-19a0-466e-95a9-2c09011aef8f" containerName="payload-processing"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788104    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b8e702-bd72-429c-a8cd-3708a10261b6" containerName="revoke-keys"
Sep 10 11:31:39.788140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.788113    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ef3b694-228c-4d98-8abb-fe5e5e13495a" containerName="main"
Sep 10 11:31:39.792415 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.792399    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:39.795276 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.795256    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:31:39.795526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.795509    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:31:39.801843 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.801810    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"]
Sep 10 11:31:39.926773 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:39.926732    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv\" (UID: \"94ecdef9-41ab-4ed7-be32-242c74c10bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:40.027450 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:40.027402    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv\" (UID: \"94ecdef9-41ab-4ed7-be32-242c74c10bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:40.028000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:40.027980    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv\" (UID: \"94ecdef9-41ab-4ed7-be32-242c74c10bc9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:40.104146 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:40.104073    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:40.378032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:40.378004    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"]
Sep 10 11:31:40.381495 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:31:40.381467    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ecdef9_41ab_4ed7_be32_242c74c10bc9.slice/crio-935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40 WatchSource:0}: Error finding container 935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40: Status 404 returned error can't find the container with id 935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40
Sep 10 11:31:40.590722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:40.590691    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv" event={"ID":"94ecdef9-41ab-4ed7-be32-242c74c10bc9","Type":"ContainerStarted","Data":"935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40"}
Sep 10 11:31:41.596941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:41.596862    2585 generic.go:356] "Generic (PLEG): container finished" podID="94ecdef9-41ab-4ed7-be32-242c74c10bc9" containerID="f50345f944c6d0e6614059d73d5e247049fb795d26e268bece17c373fe487158" exitCode=0
Sep 10 11:31:41.596941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:41.596908    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv" event={"ID":"94ecdef9-41ab-4ed7-be32-242c74c10bc9","Type":"ContainerDied","Data":"f50345f944c6d0e6614059d73d5e247049fb795d26e268bece17c373fe487158"}
Sep 10 11:31:42.734290 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:42.734260    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:42.851447 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:42.851419    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca\") pod \"94ecdef9-41ab-4ed7-be32-242c74c10bc9\" (UID: \"94ecdef9-41ab-4ed7-be32-242c74c10bc9\") "
Sep 10 11:31:42.851753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:42.851728    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94ecdef9-41ab-4ed7-be32-242c74c10bc9" (UID: "94ecdef9-41ab-4ed7-be32-242c74c10bc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:42.952952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:42.952927    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ecdef9-41ab-4ed7-be32-242c74c10bc9-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:43.608749 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:43.608711    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv" event={"ID":"94ecdef9-41ab-4ed7-be32-242c74c10bc9","Type":"ContainerDied","Data":"935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40"}
Sep 10 11:31:43.608908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:43.608762    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="935843dea539331a45d724e1e20cb996531eb011b9725976e43b450a26ae7c40"
Sep 10 11:31:43.608908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:43.608725    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"
Sep 10 11:31:50.758258 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.758226    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:31:50.758727 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.758459    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" podUID="a39ec164-fe55-467b-93e0-9d130224ecc9" containerName="payload-processing" containerID="cri-o://e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c" gracePeriod=30
Sep 10 11:31:50.766691 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.766661    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:31:50.766912 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.766885    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" podUID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" containerName="payload-pre-processing" containerID="cri-o://74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a" gracePeriod=30
Sep 10 11:31:50.977540 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.977357    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"]
Sep 10 11:31:50.979417 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:50.979386    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-4c9990-bznzv"]
Sep 10 11:31:51.042493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.042469    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:31:51.045239 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.045220    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:31:51.225187 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225149    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t26s8\" (UniqueName: \"kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8\") pod \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") "
Sep 10 11:31:51.225459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225225    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume\") pod \"a39ec164-fe55-467b-93e0-9d130224ecc9\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") "
Sep 10 11:31:51.225459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225270    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume\") pod \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\" (UID: \"e552131e-6679-43a3-a3fa-e0263ddd8ff8\") "
Sep 10 11:31:51.225459 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225358    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbxpv\" (UniqueName: \"kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv\") pod \"a39ec164-fe55-467b-93e0-9d130224ecc9\" (UID: \"a39ec164-fe55-467b-93e0-9d130224ecc9\") "
Sep 10 11:31:51.225636 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225617    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a39ec164-fe55-467b-93e0-9d130224ecc9" (UID: "a39ec164-fe55-467b-93e0-9d130224ecc9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:51.225693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.225656    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e552131e-6679-43a3-a3fa-e0263ddd8ff8" (UID: "e552131e-6679-43a3-a3fa-e0263ddd8ff8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:51.227490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.227466    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv" (OuterVolumeSpecName: "kube-api-access-sbxpv") pod "a39ec164-fe55-467b-93e0-9d130224ecc9" (UID: "a39ec164-fe55-467b-93e0-9d130224ecc9"). InnerVolumeSpecName "kube-api-access-sbxpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:51.227606 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.227558    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8" (OuterVolumeSpecName: "kube-api-access-t26s8") pod "e552131e-6679-43a3-a3fa-e0263ddd8ff8" (UID: "e552131e-6679-43a3-a3fa-e0263ddd8ff8"). InnerVolumeSpecName "kube-api-access-t26s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:51.326182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.326093    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t26s8\" (UniqueName: \"kubernetes.io/projected/e552131e-6679-43a3-a3fa-e0263ddd8ff8-kube-api-access-t26s8\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:51.326182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.326123    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a39ec164-fe55-467b-93e0-9d130224ecc9-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:51.326182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.326133    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e552131e-6679-43a3-a3fa-e0263ddd8ff8-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:51.326182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.326144    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbxpv\" (UniqueName: \"kubernetes.io/projected/a39ec164-fe55-467b-93e0-9d130224ecc9-kube-api-access-sbxpv\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:51.652406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.652292    2585 generic.go:356] "Generic (PLEG): container finished" podID="a39ec164-fe55-467b-93e0-9d130224ecc9" containerID="e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c" exitCode=0
Sep 10 11:31:51.652406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.652348    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"
Sep 10 11:31:51.652406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.652391    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" event={"ID":"a39ec164-fe55-467b-93e0-9d130224ecc9","Type":"ContainerDied","Data":"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"}
Sep 10 11:31:51.652678 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.652447    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2" event={"ID":"a39ec164-fe55-467b-93e0-9d130224ecc9","Type":"ContainerDied","Data":"903dd6e72ee4938b79a80754fefa8bf4aab5d929bae6ba53ea944f4317645e33"}
Sep 10 11:31:51.652678 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.652473    2585 scope.go:117] "RemoveContainer" containerID="e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"
Sep 10 11:31:51.653885 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.653857    2585 generic.go:356] "Generic (PLEG): container finished" podID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" containerID="74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a" exitCode=0
Sep 10 11:31:51.653970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.653904    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"
Sep 10 11:31:51.653970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.653921    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" event={"ID":"e552131e-6679-43a3-a3fa-e0263ddd8ff8","Type":"ContainerDied","Data":"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"}
Sep 10 11:31:51.653970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.653951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w" event={"ID":"e552131e-6679-43a3-a3fa-e0263ddd8ff8","Type":"ContainerDied","Data":"798b1456c38f11cdce147bafedc35c4bc7f239984c3348a09ac0932cafc6e1cf"}
Sep 10 11:31:51.669240 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.669216    2585 scope.go:117] "RemoveContainer" containerID="e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"
Sep 10 11:31:51.669580 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:51.669552    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c\": container with ID starting with e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c not found: ID does not exist" containerID="e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"
Sep 10 11:31:51.669675 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.669590    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c"} err="failed to get container status \"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c\": rpc error: code = NotFound desc = could not find container \"e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c\": container with ID starting with e11c08fc8bfc1368c66c37bd1cfcacb0f463bb49d9a2407d286caa9126da148c not found: ID does not exist"
Sep 10 11:31:51.669675 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.669614    2585 scope.go:117] "RemoveContainer" containerID="74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"
Sep 10 11:31:51.670743 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.670714    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ecdef9-41ab-4ed7-be32-242c74c10bc9" path="/var/lib/kubelet/pods/94ecdef9-41ab-4ed7-be32-242c74c10bc9/volumes"
Sep 10 11:31:51.678398 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.678379    2585 scope.go:117] "RemoveContainer" containerID="74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"
Sep 10 11:31:51.678672 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:51.678653    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a\": container with ID starting with 74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a not found: ID does not exist" containerID="74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"
Sep 10 11:31:51.678734 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.678684    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a"} err="failed to get container status \"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a\": rpc error: code = NotFound desc = could not find container \"74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a\": container with ID starting with 74d4abf9fd2de80f73b40bef540c71b9d5c41b679334d602e9c54ce0299cef5a not found: ID does not exist"
Sep 10 11:31:51.692262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.692232    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:31:51.695124 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.695101    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-4c9990-76f564bc5f-g44d2"]
Sep 10 11:31:51.709295 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.709270    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:31:51.715893 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:51.715869    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-4c9990-56698dc65d-rq86w"]
Sep 10 11:31:53.565729 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:53.565697    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:31:53.663037 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:53.663003    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a39ec164-fe55-467b-93e0-9d130224ecc9" path="/var/lib/kubelet/pods/a39ec164-fe55-467b-93e0-9d130224ecc9/volumes"
Sep 10 11:31:53.663408 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:53.663391    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" path="/var/lib/kubelet/pods/e552131e-6679-43a3-a3fa-e0263ddd8ff8/volumes"
Sep 10 11:31:57.998793 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:57.998758    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:57.999562 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:57.999507    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" podUID="68104a59-9a57-42ea-a0ef-22581b8c3a42" containerName="payload-pre-processing" containerID="cri-o://1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df" gracePeriod=30
Sep 10 11:31:58.252901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.252846    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:58.285627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.285596    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume\") pod \"68104a59-9a57-42ea-a0ef-22581b8c3a42\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") "
Sep 10 11:31:58.285824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.285642    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8cjz\" (UniqueName: \"kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz\") pod \"68104a59-9a57-42ea-a0ef-22581b8c3a42\" (UID: \"68104a59-9a57-42ea-a0ef-22581b8c3a42\") "
Sep 10 11:31:58.286012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.285986    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68104a59-9a57-42ea-a0ef-22581b8c3a42" (UID: "68104a59-9a57-42ea-a0ef-22581b8c3a42"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:31:58.287876 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.287839    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz" (OuterVolumeSpecName: "kube-api-access-h8cjz") pod "68104a59-9a57-42ea-a0ef-22581b8c3a42" (UID: "68104a59-9a57-42ea-a0ef-22581b8c3a42"). InnerVolumeSpecName "kube-api-access-h8cjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:31:58.386552 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.386517    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68104a59-9a57-42ea-a0ef-22581b8c3a42-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:58.386552 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.386544    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8cjz\" (UniqueName: \"kubernetes.io/projected/68104a59-9a57-42ea-a0ef-22581b8c3a42-kube-api-access-h8cjz\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:31:58.691542 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.691508    2585 generic.go:356] "Generic (PLEG): container finished" podID="68104a59-9a57-42ea-a0ef-22581b8c3a42" containerID="1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df" exitCode=0
Sep 10 11:31:58.691717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.691546    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"
Sep 10 11:31:58.691717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.691593    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" event={"ID":"68104a59-9a57-42ea-a0ef-22581b8c3a42","Type":"ContainerDied","Data":"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"}
Sep 10 11:31:58.691717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.691645    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt" event={"ID":"68104a59-9a57-42ea-a0ef-22581b8c3a42","Type":"ContainerDied","Data":"654e6a6d75959a342aefd6019a38953cc2424fc5551ce8c41392e62968601dca"}
Sep 10 11:31:58.691717 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.691667    2585 scope.go:117] "RemoveContainer" containerID="1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"
Sep 10 11:31:58.703015 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.702980    2585 scope.go:117] "RemoveContainer" containerID="1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"
Sep 10 11:31:58.703709 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:31:58.703677    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df\": container with ID starting with 1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df not found: ID does not exist" containerID="1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"
Sep 10 11:31:58.703820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.703708    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df"} err="failed to get container status \"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df\": rpc error: code = NotFound desc = could not find container \"1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df\": container with ID starting with 1e292fcf5b4a9113c75dc22a6c0e1fc6dadafe75acba8c1255896163658b36df not found: ID does not exist"
Sep 10 11:31:58.717815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.717791    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:58.720675 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:58.720654    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5cf411f1-7455664cd6-nrtlt"]
Sep 10 11:31:59.662660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:31:59.662629    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68104a59-9a57-42ea-a0ef-22581b8c3a42" path="/var/lib/kubelet/pods/68104a59-9a57-42ea-a0ef-22581b8c3a42/volumes"
Sep 10 11:32:02.076469 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:02.076423    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:32:05.769916 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.769885    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770221    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68104a59-9a57-42ea-a0ef-22581b8c3a42" containerName="payload-pre-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770233    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="68104a59-9a57-42ea-a0ef-22581b8c3a42" containerName="payload-pre-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770253    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" containerName="payload-pre-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770258    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" containerName="payload-pre-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770264    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a39ec164-fe55-467b-93e0-9d130224ecc9" containerName="payload-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770269    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a39ec164-fe55-467b-93e0-9d130224ecc9" containerName="payload-processing"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770282    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ecdef9-41ab-4ed7-be32-242c74c10bc9" containerName="revoke-keys"
Sep 10 11:32:05.770287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770286    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ecdef9-41ab-4ed7-be32-242c74c10bc9" containerName="revoke-keys"
Sep 10 11:32:05.770589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770348    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a39ec164-fe55-467b-93e0-9d130224ecc9" containerName="payload-processing"
Sep 10 11:32:05.770589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770358    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="e552131e-6679-43a3-a3fa-e0263ddd8ff8" containerName="payload-pre-processing"
Sep 10 11:32:05.770589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770365    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ecdef9-41ab-4ed7-be32-242c74c10bc9" containerName="revoke-keys"
Sep 10 11:32:05.770589 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.770371    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="68104a59-9a57-42ea-a0ef-22581b8c3a42" containerName="payload-pre-processing"
Sep 10 11:32:05.774472 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.774454    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.776489 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.776464    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-5fde2a-dockercfg-hm54k\""
Sep 10 11:32:05.776489 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.776469    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-5fde2a\""
Sep 10 11:32:05.784721 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.784699    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:32:05.845783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.845755    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.845955 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.845810    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcllv\" (UniqueName: \"kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.946538 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.946505    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.946705 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.946554    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcllv\" (UniqueName: \"kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.947221 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.947201    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:05.954974 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:05.954942    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcllv\" (UniqueName: \"kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv\") pod \"payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:06.084367 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.084256    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:06.361834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.361806    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:32:06.364157 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:06.364126    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda25ac598_84c8_47c9_9fb0_1d5863c99abb.slice/crio-0e9989284b9f319b802b04c785121a38f93ff03ae544ff68990efbc651022e84 WatchSource:0}: Error finding container 0e9989284b9f319b802b04c785121a38f93ff03ae544ff68990efbc651022e84: Status 404 returned error can't find the container with id 0e9989284b9f319b802b04c785121a38f93ff03ae544ff68990efbc651022e84
Sep 10 11:32:06.737987 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.737952    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" event={"ID":"a25ac598-84c8-47c9-9fb0-1d5863c99abb","Type":"ContainerStarted","Data":"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"}
Sep 10 11:32:06.738161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.738011    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" event={"ID":"a25ac598-84c8-47c9-9fb0-1d5863c99abb","Type":"ContainerStarted","Data":"0e9989284b9f319b802b04c785121a38f93ff03ae544ff68990efbc651022e84"}
Sep 10 11:32:06.738161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.738057    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:06.756752 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:06.756706    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" podStartSLOduration=1.7566907920000001 podStartE2EDuration="1.756690792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:06.755916574 +0000 UTC m=+1061.559983936" watchObservedRunningTime="2026-09-10 11:32:06.756690792 +0000 UTC m=+1061.560758139"
Sep 10 11:32:11.070005 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:11.069960    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:32:17.746236 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:17.746205    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:32:26.743233 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.743197    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:32:26.747763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.747745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.749765 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.749742    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-c2e44c\""
Sep 10 11:32:26.759710 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.759688    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:32:26.763020 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.762996    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:32:26.766336 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.766319    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.768296 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.768277    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-c2e44c-dockercfg-hlcpk\""
Sep 10 11:32:26.777716 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.777695    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:32:26.827373 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.827293    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd64k\" (UniqueName: \"kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.827555 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.827444    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq8hm\" (UniqueName: \"kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.827555 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.827469    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.827555 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.827517    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.928157 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928118    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd64k\" (UniqueName: \"kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.928364 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928186    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq8hm\" (UniqueName: \"kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.928364 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928215    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.928364 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928260    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.929000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928975    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:26.929000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.928988    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.936680 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.936656    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq8hm\" (UniqueName: \"kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm\") pod \"payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:26.936774 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:26.936729    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd64k\" (UniqueName: \"kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k\") pod \"payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:27.057888 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.057811    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:27.078183 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.078150    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:27.336621 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.336596    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:32:27.338524 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:27.338492    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc5c197_4a79_4f5c_a354_b6c4ce3cd227.slice/crio-f34896aa37aa7ac583108565f199c077354867ddc5a4ece8bc137dcd5fb66a30 WatchSource:0}: Error finding container f34896aa37aa7ac583108565f199c077354867ddc5a4ece8bc137dcd5fb66a30: Status 404 returned error can't find the container with id f34896aa37aa7ac583108565f199c077354867ddc5a4ece8bc137dcd5fb66a30
Sep 10 11:32:27.364923 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.364900    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:32:27.367078 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:27.367045    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31345535_5322_4eee_b00c_a98399ecab93.slice/crio-2d17316946c58b24c4b1317db2e691358b71939e43a19ea4dcccf20a7f2cc303 WatchSource:0}: Error finding container 2d17316946c58b24c4b1317db2e691358b71939e43a19ea4dcccf20a7f2cc303: Status 404 returned error can't find the container with id 2d17316946c58b24c4b1317db2e691358b71939e43a19ea4dcccf20a7f2cc303
Sep 10 11:32:27.852410 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.852368    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" event={"ID":"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227","Type":"ContainerStarted","Data":"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"}
Sep 10 11:32:27.852828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.852427    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" event={"ID":"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227","Type":"ContainerStarted","Data":"f34896aa37aa7ac583108565f199c077354867ddc5a4ece8bc137dcd5fb66a30"}
Sep 10 11:32:27.852828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.852468    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:27.853968 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.853940    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" event={"ID":"31345535-5322-4eee-b00c-a98399ecab93","Type":"ContainerStarted","Data":"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"}
Sep 10 11:32:27.854090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.853978    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" event={"ID":"31345535-5322-4eee-b00c-a98399ecab93","Type":"ContainerStarted","Data":"2d17316946c58b24c4b1317db2e691358b71939e43a19ea4dcccf20a7f2cc303"}
Sep 10 11:32:27.854090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.854067    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:27.871682 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.871638    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" podStartSLOduration=1.8716250410000002 podStartE2EDuration="1.871625041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:27.871436847 +0000 UTC m=+1082.675504194" watchObservedRunningTime="2026-09-10 11:32:27.871625041 +0000 UTC m=+1082.675692388"
Sep 10 11:32:27.891206 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:27.891163    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" podStartSLOduration=1.891150649 podStartE2EDuration="1.891150649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:27.890321629 +0000 UTC m=+1082.694388978" watchObservedRunningTime="2026-09-10 11:32:27.891150649 +0000 UTC m=+1082.695217997"
Sep 10 11:32:31.068624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:31.068590    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:32:34.692032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.691998    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:32:34.695621 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.695605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.697603 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.697579    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-13c392-dockercfg-ffvzb\""
Sep 10 11:32:34.697735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.697583    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-13c392\""
Sep 10 11:32:34.706734 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.706714    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:32:34.798469 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.798430    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbhvg\" (UniqueName: \"kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.798641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.798545    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.899530 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.899493    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhvg\" (UniqueName: \"kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.899765 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.899557    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.900292 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.900267    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:34.908194 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:34.908167    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhvg\" (UniqueName: \"kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg\") pod \"payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:35.008103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.008037    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:35.278526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.278501    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:32:35.280989 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:35.280963    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a6f614_c94f_4d31_8b03_c2ba51d215aa.slice/crio-9a5073646649287ad0791dde9c0fef3005a4e2d834ef9f88e2f56c07dd5a05eb WatchSource:0}: Error finding container 9a5073646649287ad0791dde9c0fef3005a4e2d834ef9f88e2f56c07dd5a05eb: Status 404 returned error can't find the container with id 9a5073646649287ad0791dde9c0fef3005a4e2d834ef9f88e2f56c07dd5a05eb
Sep 10 11:32:35.905581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.905546    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" event={"ID":"a7a6f614-c94f-4d31-8b03-c2ba51d215aa","Type":"ContainerStarted","Data":"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"}
Sep 10 11:32:35.906033 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.905607    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" event={"ID":"a7a6f614-c94f-4d31-8b03-c2ba51d215aa","Type":"ContainerStarted","Data":"9a5073646649287ad0791dde9c0fef3005a4e2d834ef9f88e2f56c07dd5a05eb"}
Sep 10 11:32:35.906033 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.905630    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:35.941872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:35.941829    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" podStartSLOduration=1.941813625 podStartE2EDuration="1.941813625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:35.937619388 +0000 UTC m=+1090.741686735" watchObservedRunningTime="2026-09-10 11:32:35.941813625 +0000 UTC m=+1090.745880972"
Sep 10 11:32:38.865816 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:38.865786    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:32:38.866267 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:38.866047    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:32:44.064604 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:44.064567    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:32:46.912116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:46.912083    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:32:56.580134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.580098    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:32:56.583508 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.583492    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.586651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.586629    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-282355\""
Sep 10 11:32:56.586785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.586733    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-282355-dockercfg-d7nfj\""
Sep 10 11:32:56.595882 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.595861    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:32:56.643317 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.643271    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:32:56.646677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.646662    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.659891 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.659862    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:32:56.683190 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.683158    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6twzt\" (UniqueName: \"kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.683347 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.683241    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.683347 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.683271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.683347 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.683287    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.784656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.784621    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6twzt\" (UniqueName: \"kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.784833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.784675    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.784833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.784693    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.784833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.784808    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.785402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.785379    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.785570 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.785549    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.793314 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.793282    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b\") pod \"payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.793417 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.793284    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6twzt\" (UniqueName: \"kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt\") pod \"payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:56.893939 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.893859    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:56.957125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:56.956888    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:57.169176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:57.169148    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:32:57.171347 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:57.171318    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11fb1178_c93a_4b32_856e_2a47d658f3aa.slice/crio-3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874 WatchSource:0}: Error finding container 3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874: Status 404 returned error can't find the container with id 3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874
Sep 10 11:32:57.234288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:57.234267    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:32:57.236281 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:32:57.236257    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ff542a_6f1a_422c_a89f_3bb11bf0a052.slice/crio-b135c10550daa6b04d8f11b452f83aafbec04dea130b503c5224e245fd1ac73b WatchSource:0}: Error finding container b135c10550daa6b04d8f11b452f83aafbec04dea130b503c5224e245fd1ac73b: Status 404 returned error can't find the container with id b135c10550daa6b04d8f11b452f83aafbec04dea130b503c5224e245fd1ac73b
Sep 10 11:32:58.023114 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.023063    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" event={"ID":"11fb1178-c93a-4b32-856e-2a47d658f3aa","Type":"ContainerStarted","Data":"ffb6b6ed3f6b480059a9d8db7121fc87b1536b30992f6a291df4edbe8b59e86c"}
Sep 10 11:32:58.023581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.023134    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" event={"ID":"11fb1178-c93a-4b32-856e-2a47d658f3aa","Type":"ContainerStarted","Data":"3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874"}
Sep 10 11:32:58.023581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.023164    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:32:58.024726 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.024696    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" event={"ID":"14ff542a-6f1a-422c-a89f-3bb11bf0a052","Type":"ContainerStarted","Data":"62159155f900f98c4d8e6b8dd22f86a0ee27bfb1bff23634c55442cad00cecf2"}
Sep 10 11:32:58.024895 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.024747    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" event={"ID":"14ff542a-6f1a-422c-a89f-3bb11bf0a052","Type":"ContainerStarted","Data":"b135c10550daa6b04d8f11b452f83aafbec04dea130b503c5224e245fd1ac73b"}
Sep 10 11:32:58.024895 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.024771    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:32:58.042672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.042619    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" podStartSLOduration=2.042604065 podStartE2EDuration="2.042604065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:58.040786253 +0000 UTC m=+1112.844853600" watchObservedRunningTime="2026-09-10 11:32:58.042604065 +0000 UTC m=+1112.846671411"
Sep 10 11:32:58.060453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:32:58.060415    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" podStartSLOduration=2.0604021 podStartE2EDuration="2.0604021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:32:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:32:58.058261256 +0000 UTC m=+1112.862328603" watchObservedRunningTime="2026-09-10 11:32:58.0604021 +0000 UTC m=+1112.864469448"
Sep 10 11:33:01.475842 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:01.475803    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:33:09.031909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:09.031875    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:33:09.032261 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:09.032052    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:33:13.276496 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.276460    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:33:13.282103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.282079    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.284605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.284572    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"openshift-service-ca.crt\""
Sep 10 11:33:13.284732 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.284578    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"default-dockercfg-4tp6l\""
Sep 10 11:33:13.284732 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.284687    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"kube-root-ca.crt\""
Sep 10 11:33:13.285175 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.285157    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"auth-test-model-5fde2a-kserve-self-signed-certs\""
Sep 10 11:33:13.288720 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.288694    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:33:13.334449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334418    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.334449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334451    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.334597 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334472    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.334597 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334493    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.334597 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.334701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.334623    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzr4w\" (UniqueName: \"kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.435848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435819    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.435993 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435852    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.435993 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435882    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.435993 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.435993 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435953    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.436198 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.435993    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzr4w\" (UniqueName: \"kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.436440 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.436412    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.436440 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.436431    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.436595 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.436473    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.438674 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.438649    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.438866 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.438848    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.444500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.444482    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzr4w\" (UniqueName: \"kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w\") pod \"auth-test-model-5fde2a-kserve-5f4d594579-5mr7j\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.593934 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.593851    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:13.868689 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:13.868602    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:33:13.871334 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:13.871289    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4cc6d8e_e9ca_4942_9d96_0e9d7538a16e.slice/crio-3bd21b3376bb761592bde11164cf32e35422807616f23ce09408764cd25559c3 WatchSource:0}: Error finding container 3bd21b3376bb761592bde11164cf32e35422807616f23ce09408764cd25559c3: Status 404 returned error can't find the container with id 3bd21b3376bb761592bde11164cf32e35422807616f23ce09408764cd25559c3
Sep 10 11:33:14.111648 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:14.111614    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" event={"ID":"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e","Type":"ContainerStarted","Data":"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"}
Sep 10 11:33:14.111648 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:14.111661    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" event={"ID":"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e","Type":"ContainerStarted","Data":"3bd21b3376bb761592bde11164cf32e35422807616f23ce09408764cd25559c3"}
Sep 10 11:33:14.130526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:14.130425    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" podStartSLOduration=1.130409443 podStartE2EDuration="1.130409443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:33:14.129199562 +0000 UTC m=+1128.933266913" watchObservedRunningTime="2026-09-10 11:33:14.130409443 +0000 UTC m=+1128.934476828"
Sep 10 11:33:16.972733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:16.972701    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:33:19.670007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.669979    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:19.673635 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.673620    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:19.676641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.676619    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cr27r\""
Sep 10 11:33:19.684230 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.684209    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:19.790898 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.790860    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l\") pod \"test-curl-7523-140591375521664\" (UID: \"1b9f8a82-3816-4fad-a315-747fd97cf469\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:19.891729 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.891691    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l\") pod \"test-curl-7523-140591375521664\" (UID: \"1b9f8a82-3816-4fad-a315-747fd97cf469\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:19.900526 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.900500    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l\") pod \"test-curl-7523-140591375521664\" (UID: \"1b9f8a82-3816-4fad-a315-747fd97cf469\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:19.983090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:19.983019    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:20.256745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:20.256719    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:20.258737 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:20.258709    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b9f8a82_3816_4fad_a315_747fd97cf469.slice/crio-c3c57e893e7e1bb6e1159f0f28ff7cec7f924149adfdd167de9feb3285daf8c1 WatchSource:0}: Error finding container c3c57e893e7e1bb6e1159f0f28ff7cec7f924149adfdd167de9feb3285daf8c1: Status 404 returned error can't find the container with id c3c57e893e7e1bb6e1159f0f28ff7cec7f924149adfdd167de9feb3285daf8c1
Sep 10 11:33:20.260139 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:20.260119    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:33:21.148906 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:21.148874    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375521664" event={"ID":"1b9f8a82-3816-4fad-a315-747fd97cf469","Type":"ContainerStarted","Data":"c3c57e893e7e1bb6e1159f0f28ff7cec7f924149adfdd167de9feb3285daf8c1"}
Sep 10 11:33:22.155012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:22.154980    2585 generic.go:356] "Generic (PLEG): container finished" podID="1b9f8a82-3816-4fad-a315-747fd97cf469" containerID="453965fbc299fe1fd1613bba3013fa37264344e58846fc6a9e0ec7c1b54a3f06" exitCode=0
Sep 10 11:33:22.155385 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:22.155073    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375521664" event={"ID":"1b9f8a82-3816-4fad-a315-747fd97cf469","Type":"ContainerDied","Data":"453965fbc299fe1fd1613bba3013fa37264344e58846fc6a9e0ec7c1b54a3f06"}
Sep 10 11:33:23.294271 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.294245    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:23.336660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.336636    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7523-140591375521664_1b9f8a82-3816-4fad-a315-747fd97cf469/test-curl-7523-140591375521664/0.log"
Sep 10 11:33:23.363022 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.362991    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:23.368346 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.366433    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:23.422645 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.422620    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l\") pod \"1b9f8a82-3816-4fad-a315-747fd97cf469\" (UID: \"1b9f8a82-3816-4fad-a315-747fd97cf469\") "
Sep 10 11:33:23.424655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.424625    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l" (OuterVolumeSpecName: "kube-api-access-dpx4l") pod "1b9f8a82-3816-4fad-a315-747fd97cf469" (UID: "1b9f8a82-3816-4fad-a315-747fd97cf469"). InnerVolumeSpecName "kube-api-access-dpx4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:33:23.494290 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.494264    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:23.494738 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.494719    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b9f8a82-3816-4fad-a315-747fd97cf469" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:23.494825 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.494740    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b9f8a82-3816-4fad-a315-747fd97cf469" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:23.494888 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.494873    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b9f8a82-3816-4fad-a315-747fd97cf469" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:23.497959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.497941    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:23.505128 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.505105    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:23.523676 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.523651    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpx4l\" (UniqueName: \"kubernetes.io/projected/1b9f8a82-3816-4fad-a315-747fd97cf469-kube-api-access-dpx4l\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:33:23.594681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.594652    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:23.594803 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.594697    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:23.607252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.607231    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:23.624034 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.624006    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rgp8\" (UniqueName: \"kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8\") pod \"test-curl-7523-140591375521664\" (UID: \"da3e4285-b5f7-4ab4-9cc4-ad960ba55598\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:23.663814 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.663786    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b9f8a82-3816-4fad-a315-747fd97cf469" path="/var/lib/kubelet/pods/1b9f8a82-3816-4fad-a315-747fd97cf469/volumes"
Sep 10 11:33:23.724728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.724662    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rgp8\" (UniqueName: \"kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8\") pod \"test-curl-7523-140591375521664\" (UID: \"da3e4285-b5f7-4ab4-9cc4-ad960ba55598\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:23.733110 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.733085    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rgp8\" (UniqueName: \"kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8\") pod \"test-curl-7523-140591375521664\" (UID: \"da3e4285-b5f7-4ab4-9cc4-ad960ba55598\") " pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:23.808191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:23.808168    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:24.077988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:24.077962    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:24.080154 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:24.080127    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podda3e4285_b5f7_4ab4_9cc4_ad960ba55598.slice/crio-76479f7063ff0c8301d47bc010c79370bb85d43c40aa97690abb91d46e5f6d0a WatchSource:0}: Error finding container 76479f7063ff0c8301d47bc010c79370bb85d43c40aa97690abb91d46e5f6d0a: Status 404 returned error can't find the container with id 76479f7063ff0c8301d47bc010c79370bb85d43c40aa97690abb91d46e5f6d0a
Sep 10 11:33:24.166262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:24.166235    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:24.166402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:24.166271    2585 scope.go:117] "RemoveContainer" containerID="453965fbc299fe1fd1613bba3013fa37264344e58846fc6a9e0ec7c1b54a3f06"
Sep 10 11:33:24.167710 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:24.167678    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375521664" event={"ID":"da3e4285-b5f7-4ab4-9cc4-ad960ba55598","Type":"ContainerStarted","Data":"76479f7063ff0c8301d47bc010c79370bb85d43c40aa97690abb91d46e5f6d0a"}
Sep 10 11:33:24.179820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:24.179787    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:33:25.175438 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:25.175397    2585 generic.go:356] "Generic (PLEG): container finished" podID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" containerID="55af57ff70937125b2499e48db7a44c3a185d2b6d3c0b38142f39a0074214ae5" exitCode=0
Sep 10 11:33:25.175707 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:25.175478    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375521664" event={"ID":"da3e4285-b5f7-4ab4-9cc4-ad960ba55598","Type":"ContainerDied","Data":"55af57ff70937125b2499e48db7a44c3a185d2b6d3c0b38142f39a0074214ae5"}
Sep 10 11:33:25.753654 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:25.753629    2585 scope.go:117] "RemoveContainer" containerID="c3806a9ffb5997c221202d0fe37d24de2f39aebcf2e8ae347f2aee75f9cbaccf"
Sep 10 11:33:26.317027 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.316994    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:26.342835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.342807    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7523-140591375521664_da3e4285-b5f7-4ab4-9cc4-ad960ba55598/test-curl-7523-140591375521664/0.log"
Sep 10 11:33:26.373104 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.373073    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:26.378651 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.378627    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7523-140591375521664"]
Sep 10 11:33:26.449067 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.449015    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rgp8\" (UniqueName: \"kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8\") pod \"da3e4285-b5f7-4ab4-9cc4-ad960ba55598\" (UID: \"da3e4285-b5f7-4ab4-9cc4-ad960ba55598\") "
Sep 10 11:33:26.451059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.451025    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8" (OuterVolumeSpecName: "kube-api-access-8rgp8") pod "da3e4285-b5f7-4ab4-9cc4-ad960ba55598" (UID: "da3e4285-b5f7-4ab4-9cc4-ad960ba55598"). InnerVolumeSpecName "kube-api-access-8rgp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:33:26.501346 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.501318    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:26.501911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.501894    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:26.501955 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.501915    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:26.502039 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.502026    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" containerName="test-curl-7523-140591375521664"
Sep 10 11:33:26.505335 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.505291    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:26.512170 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.512147    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:26.549596 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.549573    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rgp8\" (UniqueName: \"kubernetes.io/projected/da3e4285-b5f7-4ab4-9cc4-ad960ba55598-kube-api-access-8rgp8\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:33:26.650503 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.650473    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqfzw\" (UniqueName: \"kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw\") pod \"test-curl-7523-140591375522240\" (UID: \"1f2222a9-87be-4243-ba1c-6c7657eaaa56\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:26.751569 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.751490    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqfzw\" (UniqueName: \"kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw\") pod \"test-curl-7523-140591375522240\" (UID: \"1f2222a9-87be-4243-ba1c-6c7657eaaa56\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:26.759777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.759754    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqfzw\" (UniqueName: \"kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw\") pod \"test-curl-7523-140591375522240\" (UID: \"1f2222a9-87be-4243-ba1c-6c7657eaaa56\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:26.815403 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:26.815367    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:27.086214 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:27.086188    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:27.087764 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:27.087734    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f2222a9_87be_4243_ba1c_6c7657eaaa56.slice/crio-f1c96a558c9ab54867f8df55f4640d863b71e812aafef93d57302eaa5d47045f WatchSource:0}: Error finding container f1c96a558c9ab54867f8df55f4640d863b71e812aafef93d57302eaa5d47045f: Status 404 returned error can't find the container with id f1c96a558c9ab54867f8df55f4640d863b71e812aafef93d57302eaa5d47045f
Sep 10 11:33:27.187135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:27.187108    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76479f7063ff0c8301d47bc010c79370bb85d43c40aa97690abb91d46e5f6d0a"
Sep 10 11:33:27.187135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:27.187117    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:27.188248 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:27.188221    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375522240" event={"ID":"1f2222a9-87be-4243-ba1c-6c7657eaaa56","Type":"ContainerStarted","Data":"f1c96a558c9ab54867f8df55f4640d863b71e812aafef93d57302eaa5d47045f"}
Sep 10 11:33:27.197482 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:33:27.197457    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7523-140591375521664\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" pod="opendatahub/test-curl-7523-140591375521664"
Sep 10 11:33:27.663688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:27.663658    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3e4285-b5f7-4ab4-9cc4-ad960ba55598" path="/var/lib/kubelet/pods/da3e4285-b5f7-4ab4-9cc4-ad960ba55598/volumes"
Sep 10 11:33:28.196134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:28.196101    2585 generic.go:356] "Generic (PLEG): container finished" podID="1f2222a9-87be-4243-ba1c-6c7657eaaa56" containerID="2fbcd4325ee513c0636c0d71977dfb93746a47ab02e92b0e5c15401795b750fd" exitCode=0
Sep 10 11:33:28.196326 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:28.196166    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375522240" event={"ID":"1f2222a9-87be-4243-ba1c-6c7657eaaa56","Type":"ContainerDied","Data":"2fbcd4325ee513c0636c0d71977dfb93746a47ab02e92b0e5c15401795b750fd"}
Sep 10 11:33:29.335823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.335797    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:29.360439 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.360414    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7523-140591375522240_1f2222a9-87be-4243-ba1c-6c7657eaaa56/test-curl-7523-140591375522240/0.log"
Sep 10 11:33:29.385745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.385723    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:29.390554 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.390532    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:29.476656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.476586    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqfzw\" (UniqueName: \"kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw\") pod \"1f2222a9-87be-4243-ba1c-6c7657eaaa56\" (UID: \"1f2222a9-87be-4243-ba1c-6c7657eaaa56\") "
Sep 10 11:33:29.478723 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.478697    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw" (OuterVolumeSpecName: "kube-api-access-jqfzw") pod "1f2222a9-87be-4243-ba1c-6c7657eaaa56" (UID: "1f2222a9-87be-4243-ba1c-6c7657eaaa56"). InnerVolumeSpecName "kube-api-access-jqfzw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:33:29.520656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.520631    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:29.521053 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.521035    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f2222a9-87be-4243-ba1c-6c7657eaaa56" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:29.521093 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.521057    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f2222a9-87be-4243-ba1c-6c7657eaaa56" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:29.521184 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.521172    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f2222a9-87be-4243-ba1c-6c7657eaaa56" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:29.524077 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.524063    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:29.532073 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.532051    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:29.577866 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.577842    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqfzw\" (UniqueName: \"kubernetes.io/projected/1f2222a9-87be-4243-ba1c-6c7657eaaa56-kube-api-access-jqfzw\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:33:29.663408 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.663380    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f2222a9-87be-4243-ba1c-6c7657eaaa56" path="/var/lib/kubelet/pods/1f2222a9-87be-4243-ba1c-6c7657eaaa56/volumes"
Sep 10 11:33:29.678403 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.678379    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwp5j\" (UniqueName: \"kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j\") pod \"test-curl-7523-140591375522240\" (UID: \"011c01a5-cb64-4596-9841-0440b5f156a1\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:29.780017 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.779923    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwp5j\" (UniqueName: \"kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j\") pod \"test-curl-7523-140591375522240\" (UID: \"011c01a5-cb64-4596-9841-0440b5f156a1\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:29.789546 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.789515    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwp5j\" (UniqueName: \"kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j\") pod \"test-curl-7523-140591375522240\" (UID: \"011c01a5-cb64-4596-9841-0440b5f156a1\") " pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:29.833404 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:29.833376    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:30.103548 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:30.103521    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:30.106007 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:30.105981    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod011c01a5_cb64_4596_9841_0440b5f156a1.slice/crio-096bf62a6a5d4bb6ddee1f6faf30ec096b81c3b496acbab861c5409ba1b1a649 WatchSource:0}: Error finding container 096bf62a6a5d4bb6ddee1f6faf30ec096b81c3b496acbab861c5409ba1b1a649: Status 404 returned error can't find the container with id 096bf62a6a5d4bb6ddee1f6faf30ec096b81c3b496acbab861c5409ba1b1a649
Sep 10 11:33:30.207877 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:30.207852    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:30.208044 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:30.207903    2585 scope.go:117] "RemoveContainer" containerID="2fbcd4325ee513c0636c0d71977dfb93746a47ab02e92b0e5c15401795b750fd"
Sep 10 11:33:30.209175 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:30.209060    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375522240" event={"ID":"011c01a5-cb64-4596-9841-0440b5f156a1","Type":"ContainerStarted","Data":"096bf62a6a5d4bb6ddee1f6faf30ec096b81c3b496acbab861c5409ba1b1a649"}
Sep 10 11:33:31.215563 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:31.215480    2585 generic.go:356] "Generic (PLEG): container finished" podID="011c01a5-cb64-4596-9841-0440b5f156a1" containerID="c64c5b2190a516a1746c2cdb8f27d42ef7704f9a880b3ac58ef191a496f546da" exitCode=0
Sep 10 11:33:31.215902 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:31.215565    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7523-140591375522240" event={"ID":"011c01a5-cb64-4596-9841-0440b5f156a1","Type":"ContainerDied","Data":"c64c5b2190a516a1746c2cdb8f27d42ef7704f9a880b3ac58ef191a496f546da"}
Sep 10 11:33:32.352832 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.352807    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:32.376404 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.376376    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7523-140591375522240_011c01a5-cb64-4596-9841-0440b5f156a1/test-curl-7523-140591375522240/0.log"
Sep 10 11:33:32.401222 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.401196    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:32.403341 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.403315    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7523-140591375522240"]
Sep 10 11:33:32.503137 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.503061    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwp5j\" (UniqueName: \"kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j\") pod \"011c01a5-cb64-4596-9841-0440b5f156a1\" (UID: \"011c01a5-cb64-4596-9841-0440b5f156a1\") "
Sep 10 11:33:32.505062 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.505035    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j" (OuterVolumeSpecName: "kube-api-access-bwp5j") pod "011c01a5-cb64-4596-9841-0440b5f156a1" (UID: "011c01a5-cb64-4596-9841-0440b5f156a1"). InnerVolumeSpecName "kube-api-access-bwp5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:33:32.604838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:32.604801    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwp5j\" (UniqueName: \"kubernetes.io/projected/011c01a5-cb64-4596-9841-0440b5f156a1-kube-api-access-bwp5j\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:33:33.227728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:33.227698    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:33.227889 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:33.227757    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="096bf62a6a5d4bb6ddee1f6faf30ec096b81c3b496acbab861c5409ba1b1a649"
Sep 10 11:33:33.237490 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:33:33.237459    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7523-140591375522240\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="011c01a5-cb64-4596-9841-0440b5f156a1" pod="opendatahub/test-curl-7523-140591375522240"
Sep 10 11:33:33.663617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:33.663588    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="011c01a5-cb64-4596-9841-0440b5f156a1" path="/var/lib/kubelet/pods/011c01a5-cb64-4596-9841-0440b5f156a1/volumes"
Sep 10 11:33:37.977840 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:37.977807    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:33:38.182216 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.182181    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:33:38.182636 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.182621    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011c01a5-cb64-4596-9841-0440b5f156a1" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:38.182688 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.182637    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="011c01a5-cb64-4596-9841-0440b5f156a1" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:38.182723 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.182717    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="011c01a5-cb64-4596-9841-0440b5f156a1" containerName="test-curl-7523-140591375522240"
Sep 10 11:33:38.186098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.186081    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.188021 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.188000    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-52dd3973\""
Sep 10 11:33:38.188186 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.188166    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-52dd3973-dockercfg-58fbz\""
Sep 10 11:33:38.198803 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.198776    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:33:38.239690 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.239597    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:33:38.243084 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.243069    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.255218 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.255194    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:33:38.356598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.356559    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.356768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.356621    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.356768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.356662    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h8ms\" (UniqueName: \"kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.356768 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.356733    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8c4b\" (UniqueName: \"kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.457879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.457844    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.458032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.457889    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.458032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.457911    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h8ms\" (UniqueName: \"kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.458032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.457964    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8c4b\" (UniqueName: \"kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.458642 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.458620    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.458701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.458642    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.466979 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.466957    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8c4b\" (UniqueName: \"kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b\") pod \"payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") " pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.467581 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.467563    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h8ms\" (UniqueName: \"kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms\") pod \"payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.497596 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.497527    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:38.553970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.553729    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:38.771672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.771646    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:33:38.773247 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:38.773220    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db9abd2_eb8c_4f2b_a9bb_0c35ff75ede5.slice/crio-7523965bb78aa1016946502087afe2d5ee7e8d2dd00e8f0202d6ac79e757defa WatchSource:0}: Error finding container 7523965bb78aa1016946502087afe2d5ee7e8d2dd00e8f0202d6ac79e757defa: Status 404 returned error can't find the container with id 7523965bb78aa1016946502087afe2d5ee7e8d2dd00e8f0202d6ac79e757defa
Sep 10 11:33:38.842656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:38.842630    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:33:38.844558 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:33:38.844532    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855d935c_de35_48cd_87ec_d88fdcc55154.slice/crio-b9a94a888b5918f77aed5b195b7da5f36136d4ae642b032617cc67139077a652 WatchSource:0}: Error finding container b9a94a888b5918f77aed5b195b7da5f36136d4ae642b032617cc67139077a652: Status 404 returned error can't find the container with id b9a94a888b5918f77aed5b195b7da5f36136d4ae642b032617cc67139077a652
Sep 10 11:33:39.260215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.260177    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" event={"ID":"855d935c-de35-48cd-87ec-d88fdcc55154","Type":"ContainerStarted","Data":"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"}
Sep 10 11:33:39.260683 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.260233    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" event={"ID":"855d935c-de35-48cd-87ec-d88fdcc55154","Type":"ContainerStarted","Data":"b9a94a888b5918f77aed5b195b7da5f36136d4ae642b032617cc67139077a652"}
Sep 10 11:33:39.260683 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.260285    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:39.261914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.261888    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" event={"ID":"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5","Type":"ContainerStarted","Data":"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"}
Sep 10 11:33:39.262026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.261934    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" event={"ID":"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5","Type":"ContainerStarted","Data":"7523965bb78aa1016946502087afe2d5ee7e8d2dd00e8f0202d6ac79e757defa"}
Sep 10 11:33:39.262026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.261975    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:39.303293 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.303230    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" podStartSLOduration=1.303208771 podStartE2EDuration="1.303208771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:33:39.300614035 +0000 UTC m=+1154.104681384" watchObservedRunningTime="2026-09-10 11:33:39.303208771 +0000 UTC m=+1154.107276119"
Sep 10 11:33:39.327031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:39.326971    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" podStartSLOduration=1.326951973 podStartE2EDuration="1.326951973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:33:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:33:39.325835837 +0000 UTC m=+1154.129903185" watchObservedRunningTime="2026-09-10 11:33:39.326951973 +0000 UTC m=+1154.131019322"
Sep 10 11:33:50.270243 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:50.270147    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:33:50.270801 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:50.270243    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:33:54.572393 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:33:54.572355    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:34:05.963471 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:05.963431    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:34:19.769772 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:19.769732    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:34:25.678101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:25.678070    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:34:25.679146 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:25.679127    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:34:25.780524 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:25.780494    2585 scope.go:117] "RemoveContainer" containerID="4584f7200e1f51ec76a7db5ab477257a9d7ed799c188b9e1e79abae6b0202c69"
Sep 10 11:34:37.467453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:37.467413    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:34:55.674946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:34:55.674911    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:35:06.896116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:06.896083    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:35:06.896655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:06.896359    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" podUID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" containerName="payload-processing" containerID="cri-o://bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a" gracePeriod=30
Sep 10 11:35:06.912182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:06.912098    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:35:06.912406 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:06.912349    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" podUID="855d935c-de35-48cd-87ec-d88fdcc55154" containerName="payload-pre-processing" containerID="cri-o://b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da" gracePeriod=30
Sep 10 11:35:07.195015 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.194987    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:35:07.197770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.197749    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:35:07.353099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353070    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume\") pod \"855d935c-de35-48cd-87ec-d88fdcc55154\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") "
Sep 10 11:35:07.353266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353107    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume\") pod \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") "
Sep 10 11:35:07.353266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353229    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8c4b\" (UniqueName: \"kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b\") pod \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\" (UID: \"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5\") "
Sep 10 11:35:07.353389 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353267    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h8ms\" (UniqueName: \"kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms\") pod \"855d935c-de35-48cd-87ec-d88fdcc55154\" (UID: \"855d935c-de35-48cd-87ec-d88fdcc55154\") "
Sep 10 11:35:07.353501 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353481    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "855d935c-de35-48cd-87ec-d88fdcc55154" (UID: "855d935c-de35-48cd-87ec-d88fdcc55154"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:35:07.353545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353519    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" (UID: "5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:35:07.353631 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353614    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/855d935c-de35-48cd-87ec-d88fdcc55154-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:35:07.353683 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.353631    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:35:07.355486 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.355466    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms" (OuterVolumeSpecName: "kube-api-access-6h8ms") pod "855d935c-de35-48cd-87ec-d88fdcc55154" (UID: "855d935c-de35-48cd-87ec-d88fdcc55154"). InnerVolumeSpecName "kube-api-access-6h8ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:35:07.355582 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.355491    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b" (OuterVolumeSpecName: "kube-api-access-h8c4b") pod "5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" (UID: "5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5"). InnerVolumeSpecName "kube-api-access-h8c4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:35:07.454701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.454639    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8c4b\" (UniqueName: \"kubernetes.io/projected/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5-kube-api-access-h8c4b\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:35:07.454701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.454662    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h8ms\" (UniqueName: \"kubernetes.io/projected/855d935c-de35-48cd-87ec-d88fdcc55154-kube-api-access-6h8ms\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:35:07.723068 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.722992    2585 generic.go:356] "Generic (PLEG): container finished" podID="855d935c-de35-48cd-87ec-d88fdcc55154" containerID="b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da" exitCode=0
Sep 10 11:35:07.723068 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.723039    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"
Sep 10 11:35:07.723270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.723072    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" event={"ID":"855d935c-de35-48cd-87ec-d88fdcc55154","Type":"ContainerDied","Data":"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"}
Sep 10 11:35:07.723270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.723134    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr" event={"ID":"855d935c-de35-48cd-87ec-d88fdcc55154","Type":"ContainerDied","Data":"b9a94a888b5918f77aed5b195b7da5f36136d4ae642b032617cc67139077a652"}
Sep 10 11:35:07.723270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.723164    2585 scope.go:117] "RemoveContainer" containerID="b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"
Sep 10 11:35:07.724533 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.724505    2585 generic.go:356] "Generic (PLEG): container finished" podID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" containerID="bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a" exitCode=0
Sep 10 11:35:07.724639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.724549    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"
Sep 10 11:35:07.724639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.724593    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" event={"ID":"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5","Type":"ContainerDied","Data":"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"}
Sep 10 11:35:07.724733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.724642    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc" event={"ID":"5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5","Type":"ContainerDied","Data":"7523965bb78aa1016946502087afe2d5ee7e8d2dd00e8f0202d6ac79e757defa"}
Sep 10 11:35:07.732482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.732453    2585 scope.go:117] "RemoveContainer" containerID="b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"
Sep 10 11:35:07.732783 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:35:07.732763    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da\": container with ID starting with b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da not found: ID does not exist" containerID="b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"
Sep 10 11:35:07.732829 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.732791    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da"} err="failed to get container status \"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da\": rpc error: code = NotFound desc = could not find container \"b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da\": container with ID starting with b8c8e8d1cf38014e66998f6292421b4e81b2f1007ef09e822e59aec2a8c378da not found: ID does not exist"
Sep 10 11:35:07.732829 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.732806    2585 scope.go:117] "RemoveContainer" containerID="bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"
Sep 10 11:35:07.740541 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.740498    2585 scope.go:117] "RemoveContainer" containerID="bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"
Sep 10 11:35:07.741082 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:35:07.741058    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a\": container with ID starting with bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a not found: ID does not exist" containerID="bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"
Sep 10 11:35:07.741190 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.741084    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a"} err="failed to get container status \"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a\": rpc error: code = NotFound desc = could not find container \"bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a\": container with ID starting with bf5b60e5f6bb0bc2155cab671ac7b9c88e647cbb5307f7f365da0a0e54f0e40a not found: ID does not exist"
Sep 10 11:35:07.743014 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.742992    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:35:07.745935 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.745915    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-52dd3973-7bcf68cf97-dz7gc"]
Sep 10 11:35:07.757230 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.757207    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:35:07.761834 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:07.761814    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-52dd3973-dd99bc64b-ttssr"]
Sep 10 11:35:09.663775 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:09.663733    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" path="/var/lib/kubelet/pods/5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5/volumes"
Sep 10 11:35:09.664374 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:09.664257    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855d935c-de35-48cd-87ec-d88fdcc55154" path="/var/lib/kubelet/pods/855d935c-de35-48cd-87ec-d88fdcc55154/volumes"
Sep 10 11:35:16.763730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:16.763687    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:35:32.576359 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:32.576328    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:35:42.295924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.295893    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7526-f02da4"]
Sep 10 11:35:42.296397 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296286    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" containerName="payload-processing"
Sep 10 11:35:42.296397 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296325    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" containerName="payload-processing"
Sep 10 11:35:42.296397 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296337    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855d935c-de35-48cd-87ec-d88fdcc55154" containerName="payload-pre-processing"
Sep 10 11:35:42.296397 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296344    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="855d935c-de35-48cd-87ec-d88fdcc55154" containerName="payload-pre-processing"
Sep 10 11:35:42.296536 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296412    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="5db9abd2-eb8c-4f2b-a9bb-0c35ff75ede5" containerName="payload-processing"
Sep 10 11:35:42.296536 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.296423    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="855d935c-de35-48cd-87ec-d88fdcc55154" containerName="payload-pre-processing"
Sep 10 11:35:42.300751 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.300731    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:42.303113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.303090    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cr27r\""
Sep 10 11:35:42.308118 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.307848    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-f02da4"]
Sep 10 11:35:42.355770 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.355730    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hjdw\" (UniqueName: \"kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw\") pod \"mt-curl-7526-f02da4\" (UID: \"6d66af6b-091d-4ad5-9cbf-0954ee294881\") " pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:42.456253 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.456220    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hjdw\" (UniqueName: \"kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw\") pod \"mt-curl-7526-f02da4\" (UID: \"6d66af6b-091d-4ad5-9cbf-0954ee294881\") " pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:42.465006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.464977    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hjdw\" (UniqueName: \"kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw\") pod \"mt-curl-7526-f02da4\" (UID: \"6d66af6b-091d-4ad5-9cbf-0954ee294881\") " pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:42.612116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.612033    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:42.884994 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.884972    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-f02da4"]
Sep 10 11:35:42.887205 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:35:42.887166    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d66af6b_091d_4ad5_9cbf_0954ee294881.slice/crio-11e902fe36c41a6f270e6edb62e2fd06f5113db4f2236502fe19a7d921fb7c14 WatchSource:0}: Error finding container 11e902fe36c41a6f270e6edb62e2fd06f5113db4f2236502fe19a7d921fb7c14: Status 404 returned error can't find the container with id 11e902fe36c41a6f270e6edb62e2fd06f5113db4f2236502fe19a7d921fb7c14
Sep 10 11:35:42.919538 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:42.919509    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-f02da4" event={"ID":"6d66af6b-091d-4ad5-9cbf-0954ee294881","Type":"ContainerStarted","Data":"11e902fe36c41a6f270e6edb62e2fd06f5113db4f2236502fe19a7d921fb7c14"}
Sep 10 11:35:43.926878 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:43.926842    2585 generic.go:356] "Generic (PLEG): container finished" podID="6d66af6b-091d-4ad5-9cbf-0954ee294881" containerID="95c55ba5aeaf7e34b5d92c0b0eaa40bba70b1eac9f562398bc637c3212381ee2" exitCode=0
Sep 10 11:35:43.927257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:43.926929    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-f02da4" event={"ID":"6d66af6b-091d-4ad5-9cbf-0954ee294881","Type":"ContainerDied","Data":"95c55ba5aeaf7e34b5d92c0b0eaa40bba70b1eac9f562398bc637c3212381ee2"}
Sep 10 11:35:45.063494 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.063470    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:45.101291 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.101267    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7526-f02da4_6d66af6b-091d-4ad5-9cbf-0954ee294881/mt-curl-7526-f02da4/0.log"
Sep 10 11:35:45.129332 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.129288    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7526-f02da4"]
Sep 10 11:35:45.133087 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.133062    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7526-f02da4"]
Sep 10 11:35:45.179639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.179612    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hjdw\" (UniqueName: \"kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw\") pod \"6d66af6b-091d-4ad5-9cbf-0954ee294881\" (UID: \"6d66af6b-091d-4ad5-9cbf-0954ee294881\") "
Sep 10 11:35:45.181642 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.181620    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw" (OuterVolumeSpecName: "kube-api-access-4hjdw") pod "6d66af6b-091d-4ad5-9cbf-0954ee294881" (UID: "6d66af6b-091d-4ad5-9cbf-0954ee294881"). InnerVolumeSpecName "kube-api-access-4hjdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:35:45.280345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.280318    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hjdw\" (UniqueName: \"kubernetes.io/projected/6d66af6b-091d-4ad5-9cbf-0954ee294881-kube-api-access-4hjdw\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:35:45.663928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.663897    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d66af6b-091d-4ad5-9cbf-0954ee294881" path="/var/lib/kubelet/pods/6d66af6b-091d-4ad5-9cbf-0954ee294881/volumes"
Sep 10 11:35:45.941619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.941527    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-f02da4"
Sep 10 11:35:45.941619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:45.941598    2585 scope.go:117] "RemoveContainer" containerID="95c55ba5aeaf7e34b5d92c0b0eaa40bba70b1eac9f562398bc637c3212381ee2"
Sep 10 11:35:46.569800 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:46.569770    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:35:49.799646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:49.799610    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:35:56.261670 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.261637    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:35:56.262050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.262030    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d66af6b-091d-4ad5-9cbf-0954ee294881" containerName="mt-curl-7526-f02da4"
Sep 10 11:35:56.262050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.262041    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d66af6b-091d-4ad5-9cbf-0954ee294881" containerName="mt-curl-7526-f02da4"
Sep 10 11:35:56.262137 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.262111    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d66af6b-091d-4ad5-9cbf-0954ee294881" containerName="mt-curl-7526-f02da4"
Sep 10 11:35:56.264752 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.264736    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.266871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.266853    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"sub-test-model-5fde2a-kserve-self-signed-certs\""
Sep 10 11:35:56.276200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.276173    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:35:56.376148 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376115    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.376333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.376333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376187    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.376333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376243    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9hn\" (UniqueName: \"kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.376333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376274    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.376333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.376326    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.477713 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477677    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.477871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477726    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9hn\" (UniqueName: \"kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.477871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.477871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477791    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.477871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477823    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.478068 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.477872    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.478287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.478247    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.478287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.478262    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.478480 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.478343    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.480060 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.480041    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.480362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.480341    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.485603 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.485575    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9hn\" (UniqueName: \"kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn\") pod \"sub-test-model-5fde2a-kserve-86898dbf5c-fms28\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.575913 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.575846    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:35:56.850118 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.850092    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:35:56.852941 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:35:56.852912    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38c457cd_a5f9_4050_baaf_6e7b9d268518.slice/crio-8fbd014874b0ecaba5ad0bc9d83b703f3e653975212900e335b2ad21de2465aa WatchSource:0}: Error finding container 8fbd014874b0ecaba5ad0bc9d83b703f3e653975212900e335b2ad21de2465aa: Status 404 returned error can't find the container with id 8fbd014874b0ecaba5ad0bc9d83b703f3e653975212900e335b2ad21de2465aa
Sep 10 11:35:56.998907 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.998826    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" event={"ID":"38c457cd-a5f9-4050-baaf-6e7b9d268518","Type":"ContainerStarted","Data":"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"}
Sep 10 11:35:56.999098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:56.998921    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" event={"ID":"38c457cd-a5f9-4050-baaf-6e7b9d268518","Type":"ContainerStarted","Data":"8fbd014874b0ecaba5ad0bc9d83b703f3e653975212900e335b2ad21de2465aa"}
Sep 10 11:35:57.017008 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:35:57.016960    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" podStartSLOduration=1.016945092 podStartE2EDuration="1.016945092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:35:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:35:57.016325833 +0000 UTC m=+1291.820393190" watchObservedRunningTime="2026-09-10 11:35:57.016945092 +0000 UTC m=+1291.821012481"
Sep 10 11:36:01.589395 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.589330    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" podUID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" containerName="main" containerID="cri-o://4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34" gracePeriod=45
Sep 10 11:36:01.836434 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.836409    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:36:01.931590 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931560    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.931742 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931603    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzr4w\" (UniqueName: \"kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.931742 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931629    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.931822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931740    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.931822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931794    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.931946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931871    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache" (OuterVolumeSpecName: "model-cache") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:36:01.931946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931899    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir\") pod \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\" (UID: \"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e\") "
Sep 10 11:36:01.932062 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.931998    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home" (OuterVolumeSpecName: "home") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:36:01.932177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.932153    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:36:01.932295 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.932260    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:01.932295 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.932277    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:01.932295 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.932289    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:01.934055 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.934029    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm" (OuterVolumeSpecName: "dshm") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:36:01.934240 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.934215    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:36:01.934240 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:01.934229    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w" (OuterVolumeSpecName: "kube-api-access-rzr4w") pod "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" (UID: "c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e"). InnerVolumeSpecName "kube-api-access-rzr4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:36:02.029334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.029285    2585 generic.go:356] "Generic (PLEG): container finished" podID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" containerID="4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34" exitCode=0
Sep 10 11:36:02.029492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.029355    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"
Sep 10 11:36:02.029492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.029372    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" event={"ID":"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e","Type":"ContainerDied","Data":"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"}
Sep 10 11:36:02.029492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.029422    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j" event={"ID":"c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e","Type":"ContainerDied","Data":"3bd21b3376bb761592bde11164cf32e35422807616f23ce09408764cd25559c3"}
Sep 10 11:36:02.029492 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.029445    2585 scope.go:117] "RemoveContainer" containerID="4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"
Sep 10 11:36:02.033217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.033195    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:02.033217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.033214    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzr4w\" (UniqueName: \"kubernetes.io/projected/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-kube-api-access-rzr4w\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:02.033470 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.033224    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:36:02.041252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.041228    2585 scope.go:117] "RemoveContainer" containerID="4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"
Sep 10 11:36:02.041831 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:36:02.041810    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34\": container with ID starting with 4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34 not found: ID does not exist" containerID="4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"
Sep 10 11:36:02.041901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.041839    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34"} err="failed to get container status \"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34\": rpc error: code = NotFound desc = could not find container \"4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34\": container with ID starting with 4ec17bff21a6861aec8faa604f213066dfa0de85e9d0fe5678e58c431da8ca34 not found: ID does not exist"
Sep 10 11:36:02.055129 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.055104    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:36:02.062141 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:02.062112    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/auth-test-model-5fde2a-kserve-5f4d594579-5mr7j"]
Sep 10 11:36:03.187405 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:03.187371    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:36:03.662983 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:03.662951    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" path="/var/lib/kubelet/pods/c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e/volumes"
Sep 10 11:36:06.576488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:06.576459    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:36:06.576943 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:06.576510    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:36:06.588935 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:06.588908    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:36:07.067617 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:07.067588    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:36:25.824451 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:25.824419    2585 scope.go:117] "RemoveContainer" containerID="1a00538e052b586dab397fae5a2db5fe2ff166180cc74a7311ced2e1149b08a3"
Sep 10 11:36:32.686789 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:32.686745    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:36:51.963873 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:36:51.963787    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:37:01.422402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.422361    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:37:01.423000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.422978    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" containerName="main"
Sep 10 11:37:01.423090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.423003    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" containerName="main"
Sep 10 11:37:01.423145 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.423117    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4cc6d8e-e9ca-4942-9d96-0e9d7538a16e" containerName="main"
Sep 10 11:37:01.425855 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.425834    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.427985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.427965    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-e89227\""
Sep 10 11:37:01.428217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.428203    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-e89227-dockercfg-7b78p\""
Sep 10 11:37:01.437669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.437648    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:37:01.578622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.578588    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk4c8\" (UniqueName: \"kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.578805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.578755    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.680035 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.679965    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk4c8\" (UniqueName: \"kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.680177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.680063    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.680735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.680717    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.688954 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.688931    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk4c8\" (UniqueName: \"kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8\") pod \"payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") " pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:01.735758 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:01.735730    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:02.013202 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:02.013176    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:37:02.015047 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:37:02.015022    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52eb4f8e_c935_4d74_a499_04532b948cc5.slice/crio-a73c77111ca26c6a71587fb6f0edf56d007197a8c763ffb3742b8ae67b90c0d9 WatchSource:0}: Error finding container a73c77111ca26c6a71587fb6f0edf56d007197a8c763ffb3742b8ae67b90c0d9: Status 404 returned error can't find the container with id a73c77111ca26c6a71587fb6f0edf56d007197a8c763ffb3742b8ae67b90c0d9
Sep 10 11:37:02.357583 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:02.357495    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" event={"ID":"52eb4f8e-c935-4d74-a499-04532b948cc5","Type":"ContainerStarted","Data":"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"}
Sep 10 11:37:02.357583 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:02.357555    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" event={"ID":"52eb4f8e-c935-4d74-a499-04532b948cc5","Type":"ContainerStarted","Data":"a73c77111ca26c6a71587fb6f0edf56d007197a8c763ffb3742b8ae67b90c0d9"}
Sep 10 11:37:02.357763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:02.357602    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:02.378913 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:02.378868    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" podStartSLOduration=1.378854702 podStartE2EDuration="1.378854702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:37:02.377024172 +0000 UTC m=+1357.181091521" watchObservedRunningTime="2026-09-10 11:37:02.378854702 +0000 UTC m=+1357.182922046"
Sep 10 11:37:07.384666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:07.384631    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:37:13.364998 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:13.364960    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:37:15.514226 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.514195    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7526-97331f"]
Sep 10 11:37:15.518462 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.518445    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:15.520824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.520800    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cr27r\""
Sep 10 11:37:15.525160 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.523620    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-97331f"]
Sep 10 11:37:15.613116 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.613079    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf87n\" (UniqueName: \"kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n\") pod \"mt-curl-7526-97331f\" (UID: \"7e1f11da-da43-48d3-826d-4db652649849\") " pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:15.714423 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.714391    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf87n\" (UniqueName: \"kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n\") pod \"mt-curl-7526-97331f\" (UID: \"7e1f11da-da43-48d3-826d-4db652649849\") " pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:15.723958 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.723928    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf87n\" (UniqueName: \"kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n\") pod \"mt-curl-7526-97331f\" (UID: \"7e1f11da-da43-48d3-826d-4db652649849\") " pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:15.835955 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:15.835876    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:16.109198 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:16.109167    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-97331f"]
Sep 10 11:37:16.112508 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:37:16.112476    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e1f11da_da43_48d3_826d_4db652649849.slice/crio-89da9def8423f2729af991525b6253a768875c9f13aeedb2a57dd06f085f256d WatchSource:0}: Error finding container 89da9def8423f2729af991525b6253a768875c9f13aeedb2a57dd06f085f256d: Status 404 returned error can't find the container with id 89da9def8423f2729af991525b6253a768875c9f13aeedb2a57dd06f085f256d
Sep 10 11:37:16.433940 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:16.433912    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-97331f" event={"ID":"7e1f11da-da43-48d3-826d-4db652649849","Type":"ContainerStarted","Data":"89da9def8423f2729af991525b6253a768875c9f13aeedb2a57dd06f085f256d"}
Sep 10 11:37:17.441856 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:17.441822    2585 generic.go:356] "Generic (PLEG): container finished" podID="7e1f11da-da43-48d3-826d-4db652649849" containerID="70845da46435d65a51dd25d7cd8d6cbaa5de59732c0a154cb03599d04bae498d" exitCode=0
Sep 10 11:37:17.442352 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:17.441917    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-97331f" event={"ID":"7e1f11da-da43-48d3-826d-4db652649849","Type":"ContainerDied","Data":"70845da46435d65a51dd25d7cd8d6cbaa5de59732c0a154cb03599d04bae498d"}
Sep 10 11:37:18.582538 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.582515    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:18.623565 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.623534    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7526-97331f_7e1f11da-da43-48d3-826d-4db652649849/mt-curl-7526-97331f/0.log"
Sep 10 11:37:18.640763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.640733    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf87n\" (UniqueName: \"kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n\") pod \"7e1f11da-da43-48d3-826d-4db652649849\" (UID: \"7e1f11da-da43-48d3-826d-4db652649849\") "
Sep 10 11:37:18.642783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.642760    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n" (OuterVolumeSpecName: "kube-api-access-vf87n") pod "7e1f11da-da43-48d3-826d-4db652649849" (UID: "7e1f11da-da43-48d3-826d-4db652649849"). InnerVolumeSpecName "kube-api-access-vf87n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:37:18.650713 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.650687    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7526-97331f"]
Sep 10 11:37:18.655838 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.655811    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7526-97331f"]
Sep 10 11:37:18.742363 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.742236    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf87n\" (UniqueName: \"kubernetes.io/projected/7e1f11da-da43-48d3-826d-4db652649849-kube-api-access-vf87n\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:18.789547 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.789514    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7526-7b6577"]
Sep 10 11:37:18.789924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.789910    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e1f11da-da43-48d3-826d-4db652649849" containerName="mt-curl-7526-97331f"
Sep 10 11:37:18.789967 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.789925    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e1f11da-da43-48d3-826d-4db652649849" containerName="mt-curl-7526-97331f"
Sep 10 11:37:18.790047 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.789995    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e1f11da-da43-48d3-826d-4db652649849" containerName="mt-curl-7526-97331f"
Sep 10 11:37:18.793011 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.792997    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:18.801992 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.801966    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-7b6577"]
Sep 10 11:37:18.843057 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.843024    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf26v\" (UniqueName: \"kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v\") pod \"mt-curl-7526-7b6577\" (UID: \"67cdd907-dd86-43e2-8dbc-3a5347f17371\") " pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:18.944450 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.944412    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf26v\" (UniqueName: \"kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v\") pod \"mt-curl-7526-7b6577\" (UID: \"67cdd907-dd86-43e2-8dbc-3a5347f17371\") " pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:18.953854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:18.953824    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf26v\" (UniqueName: \"kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v\") pod \"mt-curl-7526-7b6577\" (UID: \"67cdd907-dd86-43e2-8dbc-3a5347f17371\") " pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:19.103503 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.103406    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:19.375796 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.375769    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7526-7b6577"]
Sep 10 11:37:19.377139 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:37:19.377109    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67cdd907_dd86_43e2_8dbc_3a5347f17371.slice/crio-7aae23e5ab3f253b9266079f3d15e9ed92acae128e71ad9883776d459418c62b WatchSource:0}: Error finding container 7aae23e5ab3f253b9266079f3d15e9ed92acae128e71ad9883776d459418c62b: Status 404 returned error can't find the container with id 7aae23e5ab3f253b9266079f3d15e9ed92acae128e71ad9883776d459418c62b
Sep 10 11:37:19.454388 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.454363    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:19.454574 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.454397    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89da9def8423f2729af991525b6253a768875c9f13aeedb2a57dd06f085f256d"
Sep 10 11:37:19.455695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.455667    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-7b6577" event={"ID":"67cdd907-dd86-43e2-8dbc-3a5347f17371","Type":"ContainerStarted","Data":"7aae23e5ab3f253b9266079f3d15e9ed92acae128e71ad9883776d459418c62b"}
Sep 10 11:37:19.464711 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:37:19.464687    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7526-97331f\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="7e1f11da-da43-48d3-826d-4db652649849" pod="opendatahub/mt-curl-7526-97331f"
Sep 10 11:37:19.663216 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:19.663183    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e1f11da-da43-48d3-826d-4db652649849" path="/var/lib/kubelet/pods/7e1f11da-da43-48d3-826d-4db652649849/volumes"
Sep 10 11:37:20.462120 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:20.462082    2585 generic.go:356] "Generic (PLEG): container finished" podID="67cdd907-dd86-43e2-8dbc-3a5347f17371" containerID="fc423d286921a7dedbaaa68733508ecd579c78e4e18ec77efeaddee1982f8cd6" exitCode=0
Sep 10 11:37:20.462332 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:20.462163    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7526-7b6577" event={"ID":"67cdd907-dd86-43e2-8dbc-3a5347f17371","Type":"ContainerDied","Data":"fc423d286921a7dedbaaa68733508ecd579c78e4e18ec77efeaddee1982f8cd6"}
Sep 10 11:37:21.602568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.602539    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:21.627388 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.627357    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7526-7b6577_67cdd907-dd86-43e2-8dbc-3a5347f17371/mt-curl-7526-7b6577/0.log"
Sep 10 11:37:21.653969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.653931    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7526-7b6577"]
Sep 10 11:37:21.664063 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.664035    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7526-7b6577"]
Sep 10 11:37:21.668960 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.668943    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bf26v\" (UniqueName: \"kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v\") pod \"67cdd907-dd86-43e2-8dbc-3a5347f17371\" (UID: \"67cdd907-dd86-43e2-8dbc-3a5347f17371\") "
Sep 10 11:37:21.670882 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.670861    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v" (OuterVolumeSpecName: "kube-api-access-bf26v") pod "67cdd907-dd86-43e2-8dbc-3a5347f17371" (UID: "67cdd907-dd86-43e2-8dbc-3a5347f17371"). InnerVolumeSpecName "kube-api-access-bf26v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:37:21.770658 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:21.770561    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf26v\" (UniqueName: \"kubernetes.io/projected/67cdd907-dd86-43e2-8dbc-3a5347f17371-kube-api-access-bf26v\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:22.473920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:22.473895    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7aae23e5ab3f253b9266079f3d15e9ed92acae128e71ad9883776d459418c62b"
Sep 10 11:37:22.473920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:22.473906    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:22.483379 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:37:22.483349    2585 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7526-7b6577\" is forbidden: User \"system:node:ip-10-0-132-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-86.ec2.internal' and this object" podUID="67cdd907-dd86-43e2-8dbc-3a5347f17371" pod="opendatahub/mt-curl-7526-7b6577"
Sep 10 11:37:23.663968 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:23.663929    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67cdd907-dd86-43e2-8dbc-3a5347f17371" path="/var/lib/kubelet/pods/67cdd907-dd86-43e2-8dbc-3a5347f17371/volumes"
Sep 10 11:37:23.816848 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:23.816816    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:37:25.852245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:25.852209    2585 scope.go:117] "RemoveContainer" containerID="dfb7bedc186c60fcf6291b0ba7adb7fe1dc5deb029efadeb674cd82eb97a1a3d"
Sep 10 11:37:26.776837 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:26.776807    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:37:30.563922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.563892    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:37:30.564320 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.564286    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67cdd907-dd86-43e2-8dbc-3a5347f17371" containerName="mt-curl-7526-7b6577"
Sep 10 11:37:30.564320 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.564314    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="67cdd907-dd86-43e2-8dbc-3a5347f17371" containerName="mt-curl-7526-7b6577"
Sep 10 11:37:30.564405 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.564386    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="67cdd907-dd86-43e2-8dbc-3a5347f17371" containerName="mt-curl-7526-7b6577"
Sep 10 11:37:30.567491 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.567473    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.572113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.572091    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-7a970e-dockercfg-bb74q\""
Sep 10 11:37:30.572257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.572238    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-7a970e\""
Sep 10 11:37:30.585136 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.585116    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:37:30.652758 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.652728    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.652912 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.652771    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9w4\" (UniqueName: \"kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.754192 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.754141    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9w4\" (UniqueName: \"kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.754401 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.754337    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.754963 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.754940    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.763403 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.763382    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9w4\" (UniqueName: \"kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4\") pod \"payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") " pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:30.876963 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:30.876896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:31.158871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:31.158798    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:37:31.161326 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:37:31.161279    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769fb802_d3be_470a_af7f_9ca034b8de7e.slice/crio-6491e42d1bd231b7cc93c1176e26dc8687ebeb75270ad3a690c44d8fddd680ba WatchSource:0}: Error finding container 6491e42d1bd231b7cc93c1176e26dc8687ebeb75270ad3a690c44d8fddd680ba: Status 404 returned error can't find the container with id 6491e42d1bd231b7cc93c1176e26dc8687ebeb75270ad3a690c44d8fddd680ba
Sep 10 11:37:31.522964 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:31.522927    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" event={"ID":"769fb802-d3be-470a-af7f-9ca034b8de7e","Type":"ContainerStarted","Data":"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"}
Sep 10 11:37:31.523185 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:31.522985    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" event={"ID":"769fb802-d3be-470a-af7f-9ca034b8de7e","Type":"ContainerStarted","Data":"6491e42d1bd231b7cc93c1176e26dc8687ebeb75270ad3a690c44d8fddd680ba"}
Sep 10 11:37:31.523185 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:31.523010    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:31.542714 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:31.542644    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" podStartSLOduration=1.542626194 podStartE2EDuration="1.542626194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:37:31.541093709 +0000 UTC m=+1386.345161057" watchObservedRunningTime="2026-09-10 11:37:31.542626194 +0000 UTC m=+1386.346693543"
Sep 10 11:37:38.836323 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:38.836263    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" podUID="38c457cd-a5f9-4050-baaf-6e7b9d268518" containerName="main" containerID="cri-o://6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d" gracePeriod=45
Sep 10 11:37:39.091030 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.090977    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:37:39.232632 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232602    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.232805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232668    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.232805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232695    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.232900 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232875    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6m9hn\" (UniqueName: \"kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.232970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232948    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:37:39.233005 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.232955    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.233089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233061    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache\") pod \"38c457cd-a5f9-4050-baaf-6e7b9d268518\" (UID: \"38c457cd-a5f9-4050-baaf-6e7b9d268518\") "
Sep 10 11:37:39.233169 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233136    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home" (OuterVolumeSpecName: "home") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:37:39.233335 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233288    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache" (OuterVolumeSpecName: "model-cache") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:37:39.233478 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233411    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.233478 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233424    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.233478 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.233437    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.234946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.234924    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:37:39.235050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.234971    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm" (OuterVolumeSpecName: "dshm") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:37:39.235050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.234981    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn" (OuterVolumeSpecName: "kube-api-access-6m9hn") pod "38c457cd-a5f9-4050-baaf-6e7b9d268518" (UID: "38c457cd-a5f9-4050-baaf-6e7b9d268518"). InnerVolumeSpecName "kube-api-access-6m9hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:37:39.334212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.334177    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38c457cd-a5f9-4050-baaf-6e7b9d268518-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.334212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.334205    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38c457cd-a5f9-4050-baaf-6e7b9d268518-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.334212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.334214    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6m9hn\" (UniqueName: \"kubernetes.io/projected/38c457cd-a5f9-4050-baaf-6e7b9d268518-kube-api-access-6m9hn\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:37:39.569011 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.568981    2585 generic.go:356] "Generic (PLEG): container finished" podID="38c457cd-a5f9-4050-baaf-6e7b9d268518" containerID="6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d" exitCode=0
Sep 10 11:37:39.569174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.569023    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" event={"ID":"38c457cd-a5f9-4050-baaf-6e7b9d268518","Type":"ContainerDied","Data":"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"}
Sep 10 11:37:39.569174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.569026    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"
Sep 10 11:37:39.569174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.569049    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28" event={"ID":"38c457cd-a5f9-4050-baaf-6e7b9d268518","Type":"ContainerDied","Data":"8fbd014874b0ecaba5ad0bc9d83b703f3e653975212900e335b2ad21de2465aa"}
Sep 10 11:37:39.569174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.569065    2585 scope.go:117] "RemoveContainer" containerID="6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"
Sep 10 11:37:39.578234 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.578216    2585 scope.go:117] "RemoveContainer" containerID="6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"
Sep 10 11:37:39.578521 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:37:39.578502    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d\": container with ID starting with 6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d not found: ID does not exist" containerID="6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"
Sep 10 11:37:39.578605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.578532    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d"} err="failed to get container status \"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d\": rpc error: code = NotFound desc = could not find container \"6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d\": container with ID starting with 6cacff5b58a7a85bb85fc1d79f78c0d66bc82f752085e85ed3100f26e7610f0d not found: ID does not exist"
Sep 10 11:37:39.590509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.590484    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:37:39.594521 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.594502    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/sub-test-model-5fde2a-kserve-86898dbf5c-fms28"]
Sep 10 11:37:39.663602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:39.663576    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38c457cd-a5f9-4050-baaf-6e7b9d268518" path="/var/lib/kubelet/pods/38c457cd-a5f9-4050-baaf-6e7b9d268518/volumes"
Sep 10 11:37:42.532677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:42.532648    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:37:43.069901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:43.069870    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:37:51.460991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.460955    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:37:51.461365 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.461357    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38c457cd-a5f9-4050-baaf-6e7b9d268518" containerName="main"
Sep 10 11:37:51.461419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.461370    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="38c457cd-a5f9-4050-baaf-6e7b9d268518" containerName="main"
Sep 10 11:37:51.461455 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.461442    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="38c457cd-a5f9-4050-baaf-6e7b9d268518" containerName="main"
Sep 10 11:37:51.465975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.465957    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.469127 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.469104    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"default-dockercfg-4tp6l\""
Sep 10 11:37:51.469257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.469157    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"kube-root-ca.crt\""
Sep 10 11:37:51.469257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.469178    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"rate-test-model-5fde2a-kserve-self-signed-certs\""
Sep 10 11:37:51.469404 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.469280    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-5fde2a\"/\"openshift-service-ca.crt\""
Sep 10 11:37:51.480995 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.480975    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:37:51.534249 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534215    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.534249 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534248    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkft\" (UniqueName: \"kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.534465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534331    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.534465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534353    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.534465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534440    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.534465 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.534456    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.634908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.634876    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.634908 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.634907    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.634946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.634985    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkft\" (UniqueName: \"kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.635029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.635058    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635382 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.635283    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635446 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.635382    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.635446 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.635394    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.637559 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.637527    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.637681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.637611    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.644211 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.644185    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkft\" (UniqueName: \"kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft\") pod \"rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") " pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:51.776297 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:51.776233    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:37:52.055172 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:52.055146    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:37:52.057124 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:37:52.057095    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7e512ed_f46c_4de0_b73a_af5631740f96.slice/crio-a984f32d3f37859098b28fec2c7c81114d5c1cc3e7b551ab6fca31016d3a4e73 WatchSource:0}: Error finding container a984f32d3f37859098b28fec2c7c81114d5c1cc3e7b551ab6fca31016d3a4e73: Status 404 returned error can't find the container with id a984f32d3f37859098b28fec2c7c81114d5c1cc3e7b551ab6fca31016d3a4e73
Sep 10 11:37:52.639498 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:52.639457    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" event={"ID":"f7e512ed-f46c-4de0-b73a-af5631740f96","Type":"ContainerStarted","Data":"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"}
Sep 10 11:37:52.639876 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:52.639508    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" event={"ID":"f7e512ed-f46c-4de0-b73a-af5631740f96","Type":"ContainerStarted","Data":"a984f32d3f37859098b28fec2c7c81114d5c1cc3e7b551ab6fca31016d3a4e73"}
Sep 10 11:37:52.660611 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:52.660554    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" podStartSLOduration=1.660538226 podStartE2EDuration="1.660538226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:37:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:37:52.658825938 +0000 UTC m=+1407.462893287" watchObservedRunningTime="2026-09-10 11:37:52.660538226 +0000 UTC m=+1407.464605575"
Sep 10 11:37:57.273310 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:37:57.273264    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:38:01.777269 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:01.777235    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:38:01.777680 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:01.777320    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:38:01.790083 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:01.790059    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:38:02.713196 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:02.713164    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:38:12.333027 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.332992    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:38:12.333470 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.333270    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" podUID="52eb4f8e-c935-4d74-a499-04532b948cc5" containerName="payload-processing" containerID="cri-o://faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe" gracePeriod=30
Sep 10 11:38:12.586532 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.586472    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:38:12.722113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.722076    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk4c8\" (UniqueName: \"kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8\") pod \"52eb4f8e-c935-4d74-a499-04532b948cc5\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") "
Sep 10 11:38:12.722344 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.722146    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume\") pod \"52eb4f8e-c935-4d74-a499-04532b948cc5\" (UID: \"52eb4f8e-c935-4d74-a499-04532b948cc5\") "
Sep 10 11:38:12.722509 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.722485    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52eb4f8e-c935-4d74-a499-04532b948cc5" (UID: "52eb4f8e-c935-4d74-a499-04532b948cc5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:38:12.724200 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.724176    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8" (OuterVolumeSpecName: "kube-api-access-kk4c8") pod "52eb4f8e-c935-4d74-a499-04532b948cc5" (UID: "52eb4f8e-c935-4d74-a499-04532b948cc5"). InnerVolumeSpecName "kube-api-access-kk4c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:38:12.754780 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.754737    2585 generic.go:356] "Generic (PLEG): container finished" podID="52eb4f8e-c935-4d74-a499-04532b948cc5" containerID="faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe" exitCode=0
Sep 10 11:38:12.754943 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.754785    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"
Sep 10 11:38:12.754943 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.754820    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" event={"ID":"52eb4f8e-c935-4d74-a499-04532b948cc5","Type":"ContainerDied","Data":"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"}
Sep 10 11:38:12.754943 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.754866    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m" event={"ID":"52eb4f8e-c935-4d74-a499-04532b948cc5","Type":"ContainerDied","Data":"a73c77111ca26c6a71587fb6f0edf56d007197a8c763ffb3742b8ae67b90c0d9"}
Sep 10 11:38:12.754943 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.754889    2585 scope.go:117] "RemoveContainer" containerID="faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"
Sep 10 11:38:12.766475 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.765642    2585 scope.go:117] "RemoveContainer" containerID="faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"
Sep 10 11:38:12.767564 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:38:12.767536    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe\": container with ID starting with faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe not found: ID does not exist" containerID="faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"
Sep 10 11:38:12.767668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.767572    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe"} err="failed to get container status \"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe\": rpc error: code = NotFound desc = could not find container \"faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe\": container with ID starting with faf0228c3736301441ea343840de0497ec1160215bfbcb20a8d8c96798f42bfe not found: ID does not exist"
Sep 10 11:38:12.778621 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.778600    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:38:12.784111 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.784083    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-e89227-7dcc68f64f-pzs2m"]
Sep 10 11:38:12.823736 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.823709    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk4c8\" (UniqueName: \"kubernetes.io/projected/52eb4f8e-c935-4d74-a499-04532b948cc5-kube-api-access-kk4c8\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:12.823736 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:12.823730    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52eb4f8e-c935-4d74-a499-04532b948cc5-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:13.663054 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:13.663021    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52eb4f8e-c935-4d74-a499-04532b948cc5" path="/var/lib/kubelet/pods/52eb4f8e-c935-4d74-a499-04532b948cc5/volumes"
Sep 10 11:38:17.480970 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.480935    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:38:17.948684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.948654    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:38:17.949052 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.949040    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52eb4f8e-c935-4d74-a499-04532b948cc5" containerName="payload-processing"
Sep 10 11:38:17.949102 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.949054    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="52eb4f8e-c935-4d74-a499-04532b948cc5" containerName="payload-processing"
Sep 10 11:38:17.949141 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.949134    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="52eb4f8e-c935-4d74-a499-04532b948cc5" containerName="payload-processing"
Sep 10 11:38:17.953467 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.953450    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:17.955669 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.955649    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-13c392\"/\"kube-root-ca.crt\""
Sep 10 11:38:17.956221 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.956200    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-13c392\"/\"rate-test-model-13c392-kserve-self-signed-certs\""
Sep 10 11:38:17.956362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.956242    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-13c392\"/\"default-dockercfg-7wps5\""
Sep 10 11:38:17.956362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.956280    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-13c392\"/\"openshift-service-ca.crt\""
Sep 10 11:38:17.962023 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:17.962002    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:38:18.074641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074579    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.074641 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074620    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.074810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074706    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9b54\" (UniqueName: \"kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.074810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074752    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.074810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074784    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.074911 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.074812    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176158 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176171    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176226    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9b54\" (UniqueName: \"kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176257    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176294    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176361 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176362    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176634 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176585    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176645    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.176695 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.176657    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.178618 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.178588    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.178761 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.178745    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.184867 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.184845    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9b54\" (UniqueName: \"kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54\") pod \"rate-test-model-13c392-kserve-6f94d885ff-jm8n7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") " pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.263387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.263287    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:18.544913 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.544888    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:38:18.546648 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:38:18.546620    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2b6f659_ebec_4590_9fc1_6019d35a0be7.slice/crio-44e79657aec282e20ef33b864fc903d52701e8e2ccb7efea7560f9aef996b005 WatchSource:0}: Error finding container 44e79657aec282e20ef33b864fc903d52701e8e2ccb7efea7560f9aef996b005: Status 404 returned error can't find the container with id 44e79657aec282e20ef33b864fc903d52701e8e2ccb7efea7560f9aef996b005
Sep 10 11:38:18.787786 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.787749    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" event={"ID":"d2b6f659-ebec-4590-9fc1-6019d35a0be7","Type":"ContainerStarted","Data":"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"}
Sep 10 11:38:18.787786 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.787790    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" event={"ID":"d2b6f659-ebec-4590-9fc1-6019d35a0be7","Type":"ContainerStarted","Data":"44e79657aec282e20ef33b864fc903d52701e8e2ccb7efea7560f9aef996b005"}
Sep 10 11:38:18.807168 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:18.807073    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" podStartSLOduration=1.807057174 podStartE2EDuration="1.807057174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:38:18.805688005 +0000 UTC m=+1433.609755363" watchObservedRunningTime="2026-09-10 11:38:18.807057174 +0000 UTC m=+1433.611124522"
Sep 10 11:38:25.872777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:25.872750    2585 scope.go:117] "RemoveContainer" containerID="f50345f944c6d0e6614059d73d5e247049fb795d26e268bece17c373fe487158"
Sep 10 11:38:28.263635 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:28.263597    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:28.264090 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:28.263649    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:28.275862 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:28.275833    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:28.852693 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:28.852664    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:38:33.073102 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:33.073068    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:38:36.193824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.193790    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:38:36.194252 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.194031    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" podUID="769fb802-d3be-470a-af7f-9ca034b8de7e" containerName="payload-processing" containerID="cri-o://abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5" gracePeriod=30
Sep 10 11:38:36.454899 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.454829    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:38:36.553237 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.553206    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb9w4\" (UniqueName: \"kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4\") pod \"769fb802-d3be-470a-af7f-9ca034b8de7e\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") "
Sep 10 11:38:36.553420 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.553263    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume\") pod \"769fb802-d3be-470a-af7f-9ca034b8de7e\" (UID: \"769fb802-d3be-470a-af7f-9ca034b8de7e\") "
Sep 10 11:38:36.553611 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.553589    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "769fb802-d3be-470a-af7f-9ca034b8de7e" (UID: "769fb802-d3be-470a-af7f-9ca034b8de7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:38:36.555340 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.555290    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4" (OuterVolumeSpecName: "kube-api-access-vb9w4") pod "769fb802-d3be-470a-af7f-9ca034b8de7e" (UID: "769fb802-d3be-470a-af7f-9ca034b8de7e"). InnerVolumeSpecName "kube-api-access-vb9w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:38:36.654619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.654576    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb9w4\" (UniqueName: \"kubernetes.io/projected/769fb802-d3be-470a-af7f-9ca034b8de7e-kube-api-access-vb9w4\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:36.654619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.654614    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769fb802-d3be-470a-af7f-9ca034b8de7e-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:36.884108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.884024    2585 generic.go:356] "Generic (PLEG): container finished" podID="769fb802-d3be-470a-af7f-9ca034b8de7e" containerID="abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5" exitCode=0
Sep 10 11:38:36.884108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.884071    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"
Sep 10 11:38:36.884288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.884068    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" event={"ID":"769fb802-d3be-470a-af7f-9ca034b8de7e","Type":"ContainerDied","Data":"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"}
Sep 10 11:38:36.884288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.884199    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld" event={"ID":"769fb802-d3be-470a-af7f-9ca034b8de7e","Type":"ContainerDied","Data":"6491e42d1bd231b7cc93c1176e26dc8687ebeb75270ad3a690c44d8fddd680ba"}
Sep 10 11:38:36.884288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.884231    2585 scope.go:117] "RemoveContainer" containerID="abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"
Sep 10 11:38:36.893671 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.893653    2585 scope.go:117] "RemoveContainer" containerID="abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"
Sep 10 11:38:36.893924 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:38:36.893904    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5\": container with ID starting with abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5 not found: ID does not exist" containerID="abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"
Sep 10 11:38:36.893972 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.893932    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5"} err="failed to get container status \"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5\": rpc error: code = NotFound desc = could not find container \"abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5\": container with ID starting with abadcddd4a067d9caffc3b1455f890c4d164b68375bc42274570e58c23fab1d5 not found: ID does not exist"
Sep 10 11:38:36.906985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.906957    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:38:36.911172 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:36.911141    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-7a970e-6d75f7b887-rdtld"]
Sep 10 11:38:37.663591 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:37.663558    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="769fb802-d3be-470a-af7f-9ca034b8de7e" path="/var/lib/kubelet/pods/769fb802-d3be-470a-af7f-9ca034b8de7e/volumes"
Sep 10 11:38:49.383683 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:49.383646    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:38:54.564442 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.564409    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:38:54.564876 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.564627    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" podUID="31345535-5322-4eee-b00c-a98399ecab93" containerName="payload-processing" containerID="cri-o://7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128" gracePeriod=30
Sep 10 11:38:54.578128 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.578099    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:38:54.578380 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.578355    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" podUID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" containerName="payload-pre-processing" containerID="cri-o://b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a" gracePeriod=30
Sep 10 11:38:54.838630 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.838602    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:38:54.853545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.853520    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:38:54.923513 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923477    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume\") pod \"31345535-5322-4eee-b00c-a98399ecab93\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") "
Sep 10 11:38:54.923694 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923534    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume\") pod \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") "
Sep 10 11:38:54.923694 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923561    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd64k\" (UniqueName: \"kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k\") pod \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\" (UID: \"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227\") "
Sep 10 11:38:54.923694 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923584    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq8hm\" (UniqueName: \"kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm\") pod \"31345535-5322-4eee-b00c-a98399ecab93\" (UID: \"31345535-5322-4eee-b00c-a98399ecab93\") "
Sep 10 11:38:54.923873 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923845    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31345535-5322-4eee-b00c-a98399ecab93" (UID: "31345535-5322-4eee-b00c-a98399ecab93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:38:54.923948 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.923922    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" (UID: "bbc5c197-4a79-4f5c-a354-b6c4ce3cd227"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:38:54.925745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.925714    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k" (OuterVolumeSpecName: "kube-api-access-zd64k") pod "bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" (UID: "bbc5c197-4a79-4f5c-a354-b6c4ce3cd227"). InnerVolumeSpecName "kube-api-access-zd64k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:38:54.925745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.925720    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm" (OuterVolumeSpecName: "kube-api-access-sq8hm") pod "31345535-5322-4eee-b00c-a98399ecab93" (UID: "31345535-5322-4eee-b00c-a98399ecab93"). InnerVolumeSpecName "kube-api-access-sq8hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:38:54.981270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.981233    2585 generic.go:356] "Generic (PLEG): container finished" podID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" containerID="b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a" exitCode=0
Sep 10 11:38:54.981488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.981279    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"
Sep 10 11:38:54.981488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.981347    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" event={"ID":"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227","Type":"ContainerDied","Data":"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"}
Sep 10 11:38:54.981488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.981406    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h" event={"ID":"bbc5c197-4a79-4f5c-a354-b6c4ce3cd227","Type":"ContainerDied","Data":"f34896aa37aa7ac583108565f199c077354867ddc5a4ece8bc137dcd5fb66a30"}
Sep 10 11:38:54.981488 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.981436    2585 scope.go:117] "RemoveContainer" containerID="b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"
Sep 10 11:38:54.982794 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.982773    2585 generic.go:356] "Generic (PLEG): container finished" podID="31345535-5322-4eee-b00c-a98399ecab93" containerID="7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128" exitCode=0
Sep 10 11:38:54.982879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.982821    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"
Sep 10 11:38:54.982879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.982841    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" event={"ID":"31345535-5322-4eee-b00c-a98399ecab93","Type":"ContainerDied","Data":"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"}
Sep 10 11:38:54.982991 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.982878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h" event={"ID":"31345535-5322-4eee-b00c-a98399ecab93","Type":"ContainerDied","Data":"2d17316946c58b24c4b1317db2e691358b71939e43a19ea4dcccf20a7f2cc303"}
Sep 10 11:38:54.991056 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.991030    2585 scope.go:117] "RemoveContainer" containerID="b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"
Sep 10 11:38:54.991386 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:38:54.991363    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a\": container with ID starting with b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a not found: ID does not exist" containerID="b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"
Sep 10 11:38:54.991499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.991397    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a"} err="failed to get container status \"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a\": rpc error: code = NotFound desc = could not find container \"b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a\": container with ID starting with b88cebff7b289e40b8ce48d939ac4288bdfd40aee9da8ac9742cdd63137c034a not found: ID does not exist"
Sep 10 11:38:54.991499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:54.991420    2585 scope.go:117] "RemoveContainer" containerID="7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"
Sep 10 11:38:55.001288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.001267    2585 scope.go:117] "RemoveContainer" containerID="7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"
Sep 10 11:38:55.001669 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:38:55.001649    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128\": container with ID starting with 7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128 not found: ID does not exist" containerID="7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"
Sep 10 11:38:55.001739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.001678    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128"} err="failed to get container status \"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128\": rpc error: code = NotFound desc = could not find container \"7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128\": container with ID starting with 7661231686d75c514f9fbd54f591fb2d5cebde38abc30128d21667275f6dd128 not found: ID does not exist"
Sep 10 11:38:55.009647 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.009617    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:38:55.017545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.017509    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c2e44c-99c996d66-whd2h"]
Sep 10 11:38:55.024638 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.024612    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31345535-5322-4eee-b00c-a98399ecab93-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:55.024638 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.024640    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:55.024783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.024650    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd64k\" (UniqueName: \"kubernetes.io/projected/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227-kube-api-access-zd64k\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:55.024783 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.024659    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq8hm\" (UniqueName: \"kubernetes.io/projected/31345535-5322-4eee-b00c-a98399ecab93-kube-api-access-sq8hm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:38:55.033106 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.033079    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:38:55.039041 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.039018    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c2e44c-85d8bd8bd7-jhn2h"]
Sep 10 11:38:55.663653 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.663623    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31345535-5322-4eee-b00c-a98399ecab93" path="/var/lib/kubelet/pods/31345535-5322-4eee-b00c-a98399ecab93/volumes"
Sep 10 11:38:55.664019 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:38:55.663995    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" path="/var/lib/kubelet/pods/bbc5c197-4a79-4f5c-a354-b6c4ce3cd227/volumes"
Sep 10 11:39:00.980835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:00.980793    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:39:02.977761 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.977729    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"]
Sep 10 11:39:02.978141 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978120    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31345535-5322-4eee-b00c-a98399ecab93" containerName="payload-processing"
Sep 10 11:39:02.978141 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978140    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="31345535-5322-4eee-b00c-a98399ecab93" containerName="payload-processing"
Sep 10 11:39:02.978223 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978158    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" containerName="payload-pre-processing"
Sep 10 11:39:02.978223 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978164    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" containerName="payload-pre-processing"
Sep 10 11:39:02.978223 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978177    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="769fb802-d3be-470a-af7f-9ca034b8de7e" containerName="payload-processing"
Sep 10 11:39:02.978223 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978182    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="769fb802-d3be-470a-af7f-9ca034b8de7e" containerName="payload-processing"
Sep 10 11:39:02.978377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978237    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="31345535-5322-4eee-b00c-a98399ecab93" containerName="payload-processing"
Sep 10 11:39:02.978377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978248    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="769fb802-d3be-470a-af7f-9ca034b8de7e" containerName="payload-processing"
Sep 10 11:39:02.978377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.978255    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc5c197-4a79-4f5c-a354-b6c4ce3cd227" containerName="payload-pre-processing"
Sep 10 11:39:02.981496 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.981481    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:02.983804 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.983786    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:39:02.984454 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.984439    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:39:02.996453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:02.996432    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"]
Sep 10 11:39:03.095320 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:03.095269    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj\" (UID: \"b92efd26-1d93-4bf8-934b-cdb7cd440b59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:03.196657 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:03.196619    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj\" (UID: \"b92efd26-1d93-4bf8-934b-cdb7cd440b59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:03.197283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:03.197255    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj\" (UID: \"b92efd26-1d93-4bf8-934b-cdb7cd440b59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:03.291583 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:03.291501    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:03.588737 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:03.588715    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"]
Sep 10 11:39:03.591560 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:39:03.591529    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92efd26_1d93_4bf8_934b_cdb7cd440b59.slice/crio-d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50 WatchSource:0}: Error finding container d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50: Status 404 returned error can't find the container with id d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50
Sep 10 11:39:04.037030 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:04.036996    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj" event={"ID":"b92efd26-1d93-4bf8-934b-cdb7cd440b59","Type":"ContainerStarted","Data":"49ea609d5ed7a274dc03b161458bbe87824aa6b3bff411b6af4c51ff8a28f616"}
Sep 10 11:39:04.037455 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:04.037071    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj" event={"ID":"b92efd26-1d93-4bf8-934b-cdb7cd440b59","Type":"ContainerStarted","Data":"d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50"}
Sep 10 11:39:04.061733 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:04.061689    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj" podStartSLOduration=2.061674862 podStartE2EDuration="2.061674862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:39:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:39:04.060437839 +0000 UTC m=+1478.864505187" watchObservedRunningTime="2026-09-10 11:39:04.061674862 +0000 UTC m=+1478.865742214"
Sep 10 11:39:05.042871 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:05.042838    2585 generic.go:356] "Generic (PLEG): container finished" podID="b92efd26-1d93-4bf8-934b-cdb7cd440b59" containerID="49ea609d5ed7a274dc03b161458bbe87824aa6b3bff411b6af4c51ff8a28f616" exitCode=0
Sep 10 11:39:05.043236 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:05.042914    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj" event={"ID":"b92efd26-1d93-4bf8-934b-cdb7cd440b59","Type":"ContainerDied","Data":"49ea609d5ed7a274dc03b161458bbe87824aa6b3bff411b6af4c51ff8a28f616"}
Sep 10 11:39:06.177075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:06.177049    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:06.327830 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:06.327739    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca\") pod \"b92efd26-1d93-4bf8-934b-cdb7cd440b59\" (UID: \"b92efd26-1d93-4bf8-934b-cdb7cd440b59\") "
Sep 10 11:39:06.328162 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:06.328127    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b92efd26-1d93-4bf8-934b-cdb7cd440b59" (UID: "b92efd26-1d93-4bf8-934b-cdb7cd440b59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:39:06.428409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:06.428378    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b92efd26-1d93-4bf8-934b-cdb7cd440b59-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:07.054739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:07.054714    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"
Sep 10 11:39:07.054739 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:07.054721    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj" event={"ID":"b92efd26-1d93-4bf8-934b-cdb7cd440b59","Type":"ContainerDied","Data":"d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50"}
Sep 10 11:39:07.054953 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:07.054770    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3a6851ea8a3f5592697167361dfd16b48fcf74d2c64fc01f64beed5914e6a50"
Sep 10 11:39:08.036514 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:08.036480    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:39:09.108093 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:09.108058    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:39:13.826857 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:13.826824    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:39:13.827282 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:13.827050    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" podUID="11fb1178-c93a-4b32-856e-2a47d658f3aa" containerName="payload-processing" containerID="cri-o://ffb6b6ed3f6b480059a9d8db7121fc87b1536b30992f6a291df4edbe8b59e86c" gracePeriod=30
Sep 10 11:39:13.839212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:13.839180    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:39:13.839493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:13.839467    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" podUID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" containerName="payload-pre-processing" containerID="cri-o://62159155f900f98c4d8e6b8dd22f86a0ee27bfb1bff23634c55442cad00cecf2" gracePeriod=30
Sep 10 11:39:14.093040 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.093009    2585 generic.go:356] "Generic (PLEG): container finished" podID="11fb1178-c93a-4b32-856e-2a47d658f3aa" containerID="ffb6b6ed3f6b480059a9d8db7121fc87b1536b30992f6a291df4edbe8b59e86c" exitCode=0
Sep 10 11:39:14.093177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.093079    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" event={"ID":"11fb1178-c93a-4b32-856e-2a47d658f3aa","Type":"ContainerDied","Data":"ffb6b6ed3f6b480059a9d8db7121fc87b1536b30992f6a291df4edbe8b59e86c"}
Sep 10 11:39:14.093177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.093139    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4" event={"ID":"11fb1178-c93a-4b32-856e-2a47d658f3aa","Type":"ContainerDied","Data":"3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874"}
Sep 10 11:39:14.093177 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.093164    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f7513630b95931705a92250291e37df5cc9e77808e80f789c1c4ba73a537874"
Sep 10 11:39:14.094563 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.094543    2585 generic.go:356] "Generic (PLEG): container finished" podID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" containerID="62159155f900f98c4d8e6b8dd22f86a0ee27bfb1bff23634c55442cad00cecf2" exitCode=0
Sep 10 11:39:14.094678 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.094591    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" event={"ID":"14ff542a-6f1a-422c-a89f-3bb11bf0a052","Type":"ContainerDied","Data":"62159155f900f98c4d8e6b8dd22f86a0ee27bfb1bff23634c55442cad00cecf2"}
Sep 10 11:39:14.114863 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.114835    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:39:14.117623 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.117604    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:39:14.131853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.131827    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"]
Sep 10 11:39:14.141662 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.141625    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-282355-8tsdj"]
Sep 10 11:39:14.300096 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300043    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume\") pod \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") "
Sep 10 11:39:14.300096 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300110    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b\") pod \"11fb1178-c93a-4b32-856e-2a47d658f3aa\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") "
Sep 10 11:39:14.300396 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300131    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume\") pod \"11fb1178-c93a-4b32-856e-2a47d658f3aa\" (UID: \"11fb1178-c93a-4b32-856e-2a47d658f3aa\") "
Sep 10 11:39:14.300396 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300168    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6twzt\" (UniqueName: \"kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt\") pod \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\" (UID: \"14ff542a-6f1a-422c-a89f-3bb11bf0a052\") "
Sep 10 11:39:14.300608 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300556    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14ff542a-6f1a-422c-a89f-3bb11bf0a052" (UID: "14ff542a-6f1a-422c-a89f-3bb11bf0a052"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:39:14.300741 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.300635    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11fb1178-c93a-4b32-856e-2a47d658f3aa" (UID: "11fb1178-c93a-4b32-856e-2a47d658f3aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:39:14.302466 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.302437    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt" (OuterVolumeSpecName: "kube-api-access-6twzt") pod "14ff542a-6f1a-422c-a89f-3bb11bf0a052" (UID: "14ff542a-6f1a-422c-a89f-3bb11bf0a052"). InnerVolumeSpecName "kube-api-access-6twzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:39:14.302466 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.302450    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b" (OuterVolumeSpecName: "kube-api-access-j6c8b") pod "11fb1178-c93a-4b32-856e-2a47d658f3aa" (UID: "11fb1178-c93a-4b32-856e-2a47d658f3aa"). InnerVolumeSpecName "kube-api-access-j6c8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:39:14.401006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.400904    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14ff542a-6f1a-422c-a89f-3bb11bf0a052-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:14.401006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.400939    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6c8b\" (UniqueName: \"kubernetes.io/projected/11fb1178-c93a-4b32-856e-2a47d658f3aa-kube-api-access-j6c8b\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:14.401006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.400952    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11fb1178-c93a-4b32-856e-2a47d658f3aa-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:14.401006 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.400964    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6twzt\" (UniqueName: \"kubernetes.io/projected/14ff542a-6f1a-422c-a89f-3bb11bf0a052-kube-api-access-6twzt\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:14.623749 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:14.623713    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:39:15.102101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.102019    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"
Sep 10 11:39:15.102101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.102040    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr" event={"ID":"14ff542a-6f1a-422c-a89f-3bb11bf0a052","Type":"ContainerDied","Data":"b135c10550daa6b04d8f11b452f83aafbec04dea130b503c5224e245fd1ac73b"}
Sep 10 11:39:15.102101 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.102095    2585 scope.go:117] "RemoveContainer" containerID="62159155f900f98c4d8e6b8dd22f86a0ee27bfb1bff23634c55442cad00cecf2"
Sep 10 11:39:15.102736 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.102019    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"
Sep 10 11:39:15.159854 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.159813    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:39:15.161482 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.161461    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-282355-58b5f97fbb-j8flr"]
Sep 10 11:39:15.189561 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.189521    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:39:15.203065 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.203031    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-282355-75dbb994d5-nm5b4"]
Sep 10 11:39:15.663626 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.663592    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11fb1178-c93a-4b32-856e-2a47d658f3aa" path="/var/lib/kubelet/pods/11fb1178-c93a-4b32-856e-2a47d658f3aa/volumes"
Sep 10 11:39:15.663985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.663961    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" path="/var/lib/kubelet/pods/14ff542a-6f1a-422c-a89f-3bb11bf0a052/volumes"
Sep 10 11:39:15.664383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.664365    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b92efd26-1d93-4bf8-934b-cdb7cd440b59" path="/var/lib/kubelet/pods/b92efd26-1d93-4bf8-934b-cdb7cd440b59/volumes"
Sep 10 11:39:15.685820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.685790    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:39:15.686153 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686132    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11fb1178-c93a-4b32-856e-2a47d658f3aa" containerName="payload-processing"
Sep 10 11:39:15.686210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686156    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="11fb1178-c93a-4b32-856e-2a47d658f3aa" containerName="payload-processing"
Sep 10 11:39:15.686210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686175    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" containerName="payload-pre-processing"
Sep 10 11:39:15.686210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686181    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" containerName="payload-pre-processing"
Sep 10 11:39:15.686210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686192    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b92efd26-1d93-4bf8-934b-cdb7cd440b59" containerName="revoke-keys"
Sep 10 11:39:15.686210 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686198    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="b92efd26-1d93-4bf8-934b-cdb7cd440b59" containerName="revoke-keys"
Sep 10 11:39:15.686380 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686256    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="11fb1178-c93a-4b32-856e-2a47d658f3aa" containerName="payload-processing"
Sep 10 11:39:15.686380 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686266    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="14ff542a-6f1a-422c-a89f-3bb11bf0a052" containerName="payload-pre-processing"
Sep 10 11:39:15.686380 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.686274    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="b92efd26-1d93-4bf8-934b-cdb7cd440b59" containerName="revoke-keys"
Sep 10 11:39:15.690735 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.690718    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.693590 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.693562    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3f4d4b\""
Sep 10 11:39:15.693697 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.693665    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3f4d4b-dockercfg-gvtkq\""
Sep 10 11:39:15.701457 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.701431    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:39:15.736386 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.736353    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:39:15.739627 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.739609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:15.759140 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.759112    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:39:15.813949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.813913    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.814098 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.813956    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wqvj\" (UniqueName: \"kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.914671 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.914579    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:15.914671 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.914619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjg9\" (UniqueName: \"kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:15.914671 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.914669    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.914957 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.914693    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wqvj\" (UniqueName: \"kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.915416 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.915394    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:15.923777 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:15.923756    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wqvj\" (UniqueName: \"kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj\") pod \"payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:16.001024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.000997    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:16.015117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.015088    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:16.015213 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.015126    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjg9\" (UniqueName: \"kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:16.016167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.016144    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:16.025151 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.025122    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjg9\" (UniqueName: \"kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9\") pod \"payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:16.049726 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.049690    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:16.285613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.285586    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:39:16.288117 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:39:16.288089    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49dd90f_0bd9_45cb_b9bb_9dad3bab31ff.slice/crio-c080d268de3c97c69de5f72faa7d5faec4f1f81d2446c0ca7c3bed456a4a5b8a WatchSource:0}: Error finding container c080d268de3c97c69de5f72faa7d5faec4f1f81d2446c0ca7c3bed456a4a5b8a: Status 404 returned error can't find the container with id c080d268de3c97c69de5f72faa7d5faec4f1f81d2446c0ca7c3bed456a4a5b8a
Sep 10 11:39:16.357745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:16.357720    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:39:16.360937 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:39:16.360894    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf1f47e_8e01_4c3a_998e_1324819e17ca.slice/crio-0ceb1012c790a8fda57ca4200bfc5c9b6333f041fe5423217450547771f4d6dc WatchSource:0}: Error finding container 0ceb1012c790a8fda57ca4200bfc5c9b6333f041fe5423217450547771f4d6dc: Status 404 returned error can't find the container with id 0ceb1012c790a8fda57ca4200bfc5c9b6333f041fe5423217450547771f4d6dc
Sep 10 11:39:17.122500 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.122465    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" event={"ID":"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff","Type":"ContainerStarted","Data":"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"}
Sep 10 11:39:17.122684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.122520    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" event={"ID":"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff","Type":"ContainerStarted","Data":"c080d268de3c97c69de5f72faa7d5faec4f1f81d2446c0ca7c3bed456a4a5b8a"}
Sep 10 11:39:17.122684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.122609    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:17.124519 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.124493    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" event={"ID":"fbf1f47e-8e01-4c3a-998e-1324819e17ca","Type":"ContainerStarted","Data":"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"}
Sep 10 11:39:17.124639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.124533    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" event={"ID":"fbf1f47e-8e01-4c3a-998e-1324819e17ca","Type":"ContainerStarted","Data":"0ceb1012c790a8fda57ca4200bfc5c9b6333f041fe5423217450547771f4d6dc"}
Sep 10 11:39:17.124639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.124616    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:17.147175 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.147120    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" podStartSLOduration=2.147103106 podStartE2EDuration="2.147103106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:39:17.144880311 +0000 UTC m=+1491.948947659" watchObservedRunningTime="2026-09-10 11:39:17.147103106 +0000 UTC m=+1491.951170455"
Sep 10 11:39:17.183271 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:17.183209    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" podStartSLOduration=2.183192646 podStartE2EDuration="2.183192646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:39:17.181106065 +0000 UTC m=+1491.985173406" watchObservedRunningTime="2026-09-10 11:39:17.183192646 +0000 UTC m=+1491.987259994"
Sep 10 11:39:23.055622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.055546    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" podUID="f7e512ed-f46c-4de0-b73a-af5631740f96" containerName="main" containerID="cri-o://ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499" gracePeriod=45
Sep 10 11:39:23.311458 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.311394    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:39:23.379718 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.379675    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.379921 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.379762    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.379921 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.379801    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkkft\" (UniqueName: \"kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.379921 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.379866    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.380224 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.380196    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:23.381920 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.381887    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm" (OuterVolumeSpecName: "dshm") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:23.382030 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.381937    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:39:23.382030 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.381986    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft" (OuterVolumeSpecName: "kube-api-access-nkkft") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "kube-api-access-nkkft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:39:23.480624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480590    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.480624 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480633    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache\") pod \"f7e512ed-f46c-4de0-b73a-af5631740f96\" (UID: \"f7e512ed-f46c-4de0-b73a-af5631740f96\") "
Sep 10 11:39:23.480833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480803    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:23.480833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480813    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:23.480833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480821    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e512ed-f46c-4de0-b73a-af5631740f96-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:23.480833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480830    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkkft\" (UniqueName: \"kubernetes.io/projected/f7e512ed-f46c-4de0-b73a-af5631740f96-kube-api-access-nkkft\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:23.480969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480898    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:23.480969 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.480905    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home" (OuterVolumeSpecName: "home") pod "f7e512ed-f46c-4de0-b73a-af5631740f96" (UID: "f7e512ed-f46c-4de0-b73a-af5631740f96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:23.581701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.581618    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:23.581701 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:23.581647    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e512ed-f46c-4de0-b73a-af5631740f96-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.127580 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.127511    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" podUID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" containerName="main" containerID="cri-o://524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b" gracePeriod=45
Sep 10 11:39:24.164112 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.164079    2585 generic.go:356] "Generic (PLEG): container finished" podID="f7e512ed-f46c-4de0-b73a-af5631740f96" containerID="ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499" exitCode=0
Sep 10 11:39:24.164288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.164117    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" event={"ID":"f7e512ed-f46c-4de0-b73a-af5631740f96","Type":"ContainerDied","Data":"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"}
Sep 10 11:39:24.164288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.164142    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"
Sep 10 11:39:24.164288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.164173    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh" event={"ID":"f7e512ed-f46c-4de0-b73a-af5631740f96","Type":"ContainerDied","Data":"a984f32d3f37859098b28fec2c7c81114d5c1cc3e7b551ab6fca31016d3a4e73"}
Sep 10 11:39:24.164288 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.164197    2585 scope.go:117] "RemoveContainer" containerID="ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"
Sep 10 11:39:24.209182 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.209145    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:39:24.213383 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.213356    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-5fde2a/rate-test-model-5fde2a-kserve-6dbf6886cf-9kwfh"]
Sep 10 11:39:24.239441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.239420    2585 scope.go:117] "RemoveContainer" containerID="ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"
Sep 10 11:39:24.239791 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:39:24.239772    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499\": container with ID starting with ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499 not found: ID does not exist" containerID="ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"
Sep 10 11:39:24.239857 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.239801    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499"} err="failed to get container status \"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499\": rpc error: code = NotFound desc = could not find container \"ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499\": container with ID starting with ce2fce0a4c53695507afc141c7f28ad1ed3033b2cc7e1cb6dc21b97281a8f499 not found: ID does not exist"
Sep 10 11:39:24.377746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.377681    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:39:24.389557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389519    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389557 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389561    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389599    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389617    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389647    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389785 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389710    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9b54\" (UniqueName: \"kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54\") pod \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\" (UID: \"d2b6f659-ebec-4590-9fc1-6019d35a0be7\") "
Sep 10 11:39:24.389984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389908    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:24.389984 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389929    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:24.390061 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.389993    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home" (OuterVolumeSpecName: "home") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:24.390103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.390072    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.390103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.390093    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.390217 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.390109    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.391936 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.391912    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:39:24.392093 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.392078    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm" (OuterVolumeSpecName: "dshm") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:39:24.392144 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.392102    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54" (OuterVolumeSpecName: "kube-api-access-f9b54") pod "d2b6f659-ebec-4590-9fc1-6019d35a0be7" (UID: "d2b6f659-ebec-4590-9fc1-6019d35a0be7"). InnerVolumeSpecName "kube-api-access-f9b54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:39:24.491163 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.491119    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2b6f659-ebec-4590-9fc1-6019d35a0be7-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.491163 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.491152    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2b6f659-ebec-4590-9fc1-6019d35a0be7-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:24.491163 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:24.491167    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9b54\" (UniqueName: \"kubernetes.io/projected/d2b6f659-ebec-4590-9fc1-6019d35a0be7-kube-api-access-f9b54\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:39:25.170914 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.170879    2585 generic.go:356] "Generic (PLEG): container finished" podID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" containerID="524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b" exitCode=0
Sep 10 11:39:25.171402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.170922    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"
Sep 10 11:39:25.171402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.170968    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" event={"ID":"d2b6f659-ebec-4590-9fc1-6019d35a0be7","Type":"ContainerDied","Data":"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"}
Sep 10 11:39:25.171402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.171019    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7" event={"ID":"d2b6f659-ebec-4590-9fc1-6019d35a0be7","Type":"ContainerDied","Data":"44e79657aec282e20ef33b864fc903d52701e8e2ccb7efea7560f9aef996b005"}
Sep 10 11:39:25.171402 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.171045    2585 scope.go:117] "RemoveContainer" containerID="524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"
Sep 10 11:39:25.180737 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.180717    2585 scope.go:117] "RemoveContainer" containerID="524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"
Sep 10 11:39:25.181009 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:39:25.180985    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b\": container with ID starting with 524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b not found: ID does not exist" containerID="524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"
Sep 10 11:39:25.181048 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.181020    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b"} err="failed to get container status \"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b\": rpc error: code = NotFound desc = could not find container \"524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b\": container with ID starting with 524a356babe9a82e975646bcabde3166f38f66f7c30727b0d2592ba249751f1b not found: ID does not exist"
Sep 10 11:39:25.192759 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.192734    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:39:25.196877 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.196857    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-13c392/rate-test-model-13c392-kserve-6f94d885ff-jm8n7"]
Sep 10 11:39:25.474344 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.474242    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:39:25.664016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.663982    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" path="/var/lib/kubelet/pods/d2b6f659-ebec-4590-9fc1-6019d35a0be7/volumes"
Sep 10 11:39:25.664388 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.664375    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e512ed-f46c-4de0-b73a-af5631740f96" path="/var/lib/kubelet/pods/f7e512ed-f46c-4de0-b73a-af5631740f96/volumes"
Sep 10 11:39:25.710517 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.710482    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:39:25.713103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.713083    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:39:25.902145 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.902038    2585 scope.go:117] "RemoveContainer" containerID="55af57ff70937125b2499e48db7a44c3a185d2b6d3c0b38142f39a0074214ae5"
Sep 10 11:39:25.910747 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:25.910727    2585 scope.go:117] "RemoveContainer" containerID="ffb6b6ed3f6b480059a9d8db7121fc87b1536b30992f6a291df4edbe8b59e86c"
Sep 10 11:39:28.131691 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:28.131659    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:39:28.132179 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:28.131740    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:39:44.775374 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775339    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:39:44.775839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775742    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e512ed-f46c-4de0-b73a-af5631740f96" containerName="main"
Sep 10 11:39:44.775839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775755    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e512ed-f46c-4de0-b73a-af5631740f96" containerName="main"
Sep 10 11:39:44.775839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775770    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" containerName="main"
Sep 10 11:39:44.775839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775776    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" containerName="main"
Sep 10 11:39:44.775839 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775839    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2b6f659-ebec-4590-9fc1-6019d35a0be7" containerName="main"
Sep 10 11:39:44.776084 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.775849    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e512ed-f46c-4de0-b73a-af5631740f96" containerName="main"
Sep 10 11:39:44.779025 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.779008    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.784385 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.784358    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-420024-dockercfg-4qgwb\""
Sep 10 11:39:44.785494 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.785315    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-420024\""
Sep 10 11:39:44.794681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.794659    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:39:44.819321 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.819271    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:39:44.822930 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.822905    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.836529 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.836502    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:39:44.872875 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.872845    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-787nx\" (UniqueName: \"kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.873037 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.872883    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmtsp\" (UniqueName: \"kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.873037 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.872919    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.873037 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.872952    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.973730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.973696    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-787nx\" (UniqueName: \"kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.973730 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.973739    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmtsp\" (UniqueName: \"kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.973952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.973783    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.973952 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.973804    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.974598 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.974579    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.974656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.974600    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:44.984763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.984731    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-787nx\" (UniqueName: \"kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx\") pod \"payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:44.984763 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:44.984742    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmtsp\" (UniqueName: \"kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp\") pod \"payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:45.089290 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:45.089195    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:45.133629 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:45.133595    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:45.375654 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:45.375516    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:39:45.378099 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:39:45.378069    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4df18f_9bcb_4ccf_9407_d493845b5d53.slice/crio-267acc212cc4b007837a03ead4da69c2bea61c9825a3ea5efd5609fa760e1316 WatchSource:0}: Error finding container 267acc212cc4b007837a03ead4da69c2bea61c9825a3ea5efd5609fa760e1316: Status 404 returned error can't find the container with id 267acc212cc4b007837a03ead4da69c2bea61c9825a3ea5efd5609fa760e1316
Sep 10 11:39:45.438476 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:45.438451    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:39:45.445024 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:39:45.444998    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc07336e_3ab0_4b3b_a537_755baed2878c.slice/crio-f1a03564792677c0361ecc278d41b2f83f4b45a89cb47a18f1cc649b4989aa63 WatchSource:0}: Error finding container f1a03564792677c0361ecc278d41b2f83f4b45a89cb47a18f1cc649b4989aa63: Status 404 returned error can't find the container with id f1a03564792677c0361ecc278d41b2f83f4b45a89cb47a18f1cc649b4989aa63
Sep 10 11:39:46.291467 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.291427    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" event={"ID":"1d4df18f-9bcb-4ccf-9407-d493845b5d53","Type":"ContainerStarted","Data":"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"}
Sep 10 11:39:46.291922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.291524    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:46.291922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.291549    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" event={"ID":"1d4df18f-9bcb-4ccf-9407-d493845b5d53","Type":"ContainerStarted","Data":"267acc212cc4b007837a03ead4da69c2bea61c9825a3ea5efd5609fa760e1316"}
Sep 10 11:39:46.293001 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.292979    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" event={"ID":"cc07336e-3ab0-4b3b-a537-755baed2878c","Type":"ContainerStarted","Data":"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"}
Sep 10 11:39:46.293081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.293019    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" event={"ID":"cc07336e-3ab0-4b3b-a537-755baed2878c","Type":"ContainerStarted","Data":"f1a03564792677c0361ecc278d41b2f83f4b45a89cb47a18f1cc649b4989aa63"}
Sep 10 11:39:46.293120 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.293074    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:39:46.312103 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.312053    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" podStartSLOduration=2.312038026 podStartE2EDuration="2.312038026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:39:46.310597012 +0000 UTC m=+1521.114664360" watchObservedRunningTime="2026-09-10 11:39:46.312038026 +0000 UTC m=+1521.116105373"
Sep 10 11:39:46.357998 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:46.357944    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" podStartSLOduration=2.357926728 podStartE2EDuration="2.357926728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:39:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:39:46.356429933 +0000 UTC m=+1521.160497280" watchObservedRunningTime="2026-09-10 11:39:46.357926728 +0000 UTC m=+1521.161994107"
Sep 10 11:39:49.930113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:49.930033    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:39:57.300743 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:57.300710    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:39:57.303161 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:39:57.300795    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:40:13.075971 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:13.075934    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk_cc07336e-3ab0-4b3b-a537-755baed2878c/payload-processing/0.log"
Sep 10 11:40:18.957124 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.957079    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:40:18.961072 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.961045    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.963362 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.963341    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3f4d4b\"/\"kube-root-ca.crt\""
Sep 10 11:40:18.963765 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.963745    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3f4d4b\"/\"ipp-route-3f4d4b-kserve-self-signed-certs\""
Sep 10 11:40:18.963821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.963780    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-3f4d4b\"/\"openshift-service-ca.crt\""
Sep 10 11:40:18.963821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.963810    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-3f4d4b\"/\"default-dockercfg-xpp88\""
Sep 10 11:40:18.970322 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.970282    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:40:18.992682 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992656    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.992818 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.992818 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992730    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.992818 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.992924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992815    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:18.992924 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:18.992842    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7dh\" (UniqueName: \"kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094248 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094195    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094248 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094249    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094384    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094487    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094523    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7dh\" (UniqueName: \"kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094628    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094689    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.094828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.094775    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.096737 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.096719    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.097045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.097026    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.105419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.105392    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7dh\" (UniqueName: \"kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh\") pod \"ipp-route-3f4d4b-kserve-7c857cf5ff-n856j\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") " pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.272190 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.272096    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:19.550602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:19.550518    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:40:19.553451 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:40:19.553420    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09c14362_86c6_490b_9bcb_acc28c3c22af.slice/crio-67a94ec76075e2cbbfe88bb5baa7f920e63b8d2b6c405850dbde275788f91d57 WatchSource:0}: Error finding container 67a94ec76075e2cbbfe88bb5baa7f920e63b8d2b6c405850dbde275788f91d57: Status 404 returned error can't find the container with id 67a94ec76075e2cbbfe88bb5baa7f920e63b8d2b6c405850dbde275788f91d57
Sep 10 11:40:20.467061 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:20.467022    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" event={"ID":"09c14362-86c6-490b-9bcb-acc28c3c22af","Type":"ContainerStarted","Data":"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"}
Sep 10 11:40:20.467061 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:20.467074    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" event={"ID":"09c14362-86c6-490b-9bcb-acc28c3c22af","Type":"ContainerStarted","Data":"67a94ec76075e2cbbfe88bb5baa7f920e63b8d2b6c405850dbde275788f91d57"}
Sep 10 11:40:20.490861 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:20.490808    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" podStartSLOduration=2.490793374 podStartE2EDuration="2.490793374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:40:20.488105599 +0000 UTC m=+1555.292172949" watchObservedRunningTime="2026-09-10 11:40:20.490793374 +0000 UTC m=+1555.294860764"
Sep 10 11:40:22.970032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:22.969996    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:40:25.957480 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:25.957445    2585 scope.go:117] "RemoveContainer" containerID="c64c5b2190a516a1746c2cdb8f27d42ef7704f9a880b3ac58ef191a496f546da"
Sep 10 11:40:29.272431 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:29.272385    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:29.272824 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:29.272450    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:29.285082 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:29.285056    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:29.525879 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:29.525792    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:40:38.266990 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:38.266953    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:40:43.644379 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:43.644320    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6_fbf1f47e-8e01-4c3a-998e-1324819e17ca/payload-processing/0.log"
Sep 10 11:40:43.901741 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:43.901642    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk_cc07336e-3ab0-4b3b-a537-755baed2878c/payload-processing/0.log"
Sep 10 11:40:49.732748 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.732712    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:40:49.736365 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.736347    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.738820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.738798    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-34501f-dockercfg-2tr4l\""
Sep 10 11:40:49.739360 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.739344    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-34501f\""
Sep 10 11:40:49.750608 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.750580    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:40:49.874398 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.874364    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.874584 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.874412    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcns4\" (UniqueName: \"kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.975334 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.975270    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.975507 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.975369    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcns4\" (UniqueName: \"kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.976027 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.976004    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:49.991575 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:49.991505    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcns4\" (UniqueName: \"kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4\") pod \"payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:50.046684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.046647    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:50.330130 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.330057    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:40:50.334089 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:40:50.334067    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ac628c_4c68_480b_8518_6b1446cd7329.slice/crio-a118b72f537be70b87ebfe86278c4e99f2d5c7d41d1026d2fb10ba159c3db7c8 WatchSource:0}: Error finding container a118b72f537be70b87ebfe86278c4e99f2d5c7d41d1026d2fb10ba159c3db7c8: Status 404 returned error can't find the container with id a118b72f537be70b87ebfe86278c4e99f2d5c7d41d1026d2fb10ba159c3db7c8
Sep 10 11:40:50.622105 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.622007    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" event={"ID":"76ac628c-4c68-480b-8518-6b1446cd7329","Type":"ContainerStarted","Data":"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"}
Sep 10 11:40:50.622266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.622108    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" event={"ID":"76ac628c-4c68-480b-8518-6b1446cd7329","Type":"ContainerStarted","Data":"a118b72f537be70b87ebfe86278c4e99f2d5c7d41d1026d2fb10ba159c3db7c8"}
Sep 10 11:40:50.622266 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.622145    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:40:50.646289 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:50.646233    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" podStartSLOduration=1.646216769 podStartE2EDuration="1.646216769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:40:50.645569235 +0000 UTC m=+1585.449636590" watchObservedRunningTime="2026-09-10 11:40:50.646216769 +0000 UTC m=+1585.450284116"
Sep 10 11:40:57.074085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:40:57.074048    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:41:01.629229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:01.629197    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:41:13.514467 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.514429    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"]
Sep 10 11:41:13.518235 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.518217    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:13.520801 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.520768    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:41:13.520928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.520815    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:41:13.527113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.527085    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"]
Sep 10 11:41:13.586205 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.586165    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8\" (UID: \"67ff8f9f-902a-44c0-be85-013aceddcd96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:13.686798 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.686768    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8\" (UID: \"67ff8f9f-902a-44c0-be85-013aceddcd96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:13.687408 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.687391    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8\" (UID: \"67ff8f9f-902a-44c0-be85-013aceddcd96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:13.830143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:13.830052    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:14.102143 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:14.102116    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"]
Sep 10 11:41:14.104033 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:41:14.104003    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ff8f9f_902a_44c0_be85_013aceddcd96.slice/crio-3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde WatchSource:0}: Error finding container 3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde: Status 404 returned error can't find the container with id 3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde
Sep 10 11:41:14.749961 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:14.749925    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8" event={"ID":"67ff8f9f-902a-44c0-be85-013aceddcd96","Type":"ContainerStarted","Data":"0ad163b65116bc0fad956b5f87a5b1a9d777913a15bb8b2e4ab0d983bbc367b8"}
Sep 10 11:41:14.750363 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:14.749982    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8" event={"ID":"67ff8f9f-902a-44c0-be85-013aceddcd96","Type":"ContainerStarted","Data":"3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde"}
Sep 10 11:41:14.768918 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:14.768874    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8" podStartSLOduration=1.768858685 podStartE2EDuration="1.768858685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:41:14.7685735 +0000 UTC m=+1609.572640848" watchObservedRunningTime="2026-09-10 11:41:14.768858685 +0000 UTC m=+1609.572926032"
Sep 10 11:41:15.755715 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:15.755679    2585 generic.go:356] "Generic (PLEG): container finished" podID="67ff8f9f-902a-44c0-be85-013aceddcd96" containerID="0ad163b65116bc0fad956b5f87a5b1a9d777913a15bb8b2e4ab0d983bbc367b8" exitCode=0
Sep 10 11:41:15.756109 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:15.755764    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8" event={"ID":"67ff8f9f-902a-44c0-be85-013aceddcd96","Type":"ContainerDied","Data":"0ad163b65116bc0fad956b5f87a5b1a9d777913a15bb8b2e4ab0d983bbc367b8"}
Sep 10 11:41:16.886167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:16.886136    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:16.917487 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:16.917452    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca\") pod \"67ff8f9f-902a-44c0-be85-013aceddcd96\" (UID: \"67ff8f9f-902a-44c0-be85-013aceddcd96\") "
Sep 10 11:41:16.917872 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:16.917847    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67ff8f9f-902a-44c0-be85-013aceddcd96" (UID: "67ff8f9f-902a-44c0-be85-013aceddcd96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:41:17.018664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:17.018594    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67ff8f9f-902a-44c0-be85-013aceddcd96-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:17.766415 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:17.766385    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"
Sep 10 11:41:17.766415 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:17.766399    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8" event={"ID":"67ff8f9f-902a-44c0-be85-013aceddcd96","Type":"ContainerDied","Data":"3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde"}
Sep 10 11:41:17.766602 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:17.766440    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c422d4884b06c6836c9a0fbf3700c4b38ba4019f193e418173dc462a2284fde"
Sep 10 11:41:19.884560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:19.884521    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:41:19.885059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:19.884777    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" podUID="76ac628c-4c68-480b-8518-6b1446cd7329" containerName="payload-processing" containerID="cri-o://71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15" gracePeriod=30
Sep 10 11:41:20.081654 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.081583    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"]
Sep 10 11:41:20.083937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.083910    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-34501f-k52w8"]
Sep 10 11:41:20.145062 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.145003    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:41:20.246318 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.246270    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume\") pod \"76ac628c-4c68-480b-8518-6b1446cd7329\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") "
Sep 10 11:41:20.246497 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.246359    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcns4\" (UniqueName: \"kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4\") pod \"76ac628c-4c68-480b-8518-6b1446cd7329\" (UID: \"76ac628c-4c68-480b-8518-6b1446cd7329\") "
Sep 10 11:41:20.246653 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.246628    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76ac628c-4c68-480b-8518-6b1446cd7329" (UID: "76ac628c-4c68-480b-8518-6b1446cd7329"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:41:20.248393 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.248372    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4" (OuterVolumeSpecName: "kube-api-access-rcns4") pod "76ac628c-4c68-480b-8518-6b1446cd7329" (UID: "76ac628c-4c68-480b-8518-6b1446cd7329"). InnerVolumeSpecName "kube-api-access-rcns4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:41:20.347808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.347771    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76ac628c-4c68-480b-8518-6b1446cd7329-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:20.347808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.347805    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcns4\" (UniqueName: \"kubernetes.io/projected/76ac628c-4c68-480b-8518-6b1446cd7329-kube-api-access-rcns4\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:20.781930 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.781894    2585 generic.go:356] "Generic (PLEG): container finished" podID="76ac628c-4c68-480b-8518-6b1446cd7329" containerID="71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15" exitCode=0
Sep 10 11:41:20.782224 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.781953    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"
Sep 10 11:41:20.782224 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.781975    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" event={"ID":"76ac628c-4c68-480b-8518-6b1446cd7329","Type":"ContainerDied","Data":"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"}
Sep 10 11:41:20.782224 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.782031    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc" event={"ID":"76ac628c-4c68-480b-8518-6b1446cd7329","Type":"ContainerDied","Data":"a118b72f537be70b87ebfe86278c4e99f2d5c7d41d1026d2fb10ba159c3db7c8"}
Sep 10 11:41:20.782224 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.782054    2585 scope.go:117] "RemoveContainer" containerID="71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"
Sep 10 11:41:20.791039 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.791020    2585 scope.go:117] "RemoveContainer" containerID="71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"
Sep 10 11:41:20.791332 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:41:20.791289    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15\": container with ID starting with 71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15 not found: ID does not exist" containerID="71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"
Sep 10 11:41:20.791409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.791338    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15"} err="failed to get container status \"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15\": rpc error: code = NotFound desc = could not find container \"71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15\": container with ID starting with 71ae6e3c024f4497fc19aaa4ad8f9f7411b087022b69a265a2808d6bfb977f15 not found: ID does not exist"
Sep 10 11:41:20.803563 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.803545    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:41:20.808067 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:20.808041    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-34501f-cdbd85ff7-grbxc"]
Sep 10 11:41:21.665605 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:21.665571    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67ff8f9f-902a-44c0-be85-013aceddcd96" path="/var/lib/kubelet/pods/67ff8f9f-902a-44c0-be85-013aceddcd96/volumes"
Sep 10 11:41:21.666081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:21.666060    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76ac628c-4c68-480b-8518-6b1446cd7329" path="/var/lib/kubelet/pods/76ac628c-4c68-480b-8518-6b1446cd7329/volumes"
Sep 10 11:41:23.980279 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:23.980243    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:41:30.929174 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929141    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"]
Sep 10 11:41:30.929604 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929586    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ac628c-4c68-480b-8518-6b1446cd7329" containerName="payload-processing"
Sep 10 11:41:30.929655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929606    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ac628c-4c68-480b-8518-6b1446cd7329" containerName="payload-processing"
Sep 10 11:41:30.929655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929628    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ff8f9f-902a-44c0-be85-013aceddcd96" containerName="revoke-keys"
Sep 10 11:41:30.929655 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929635    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ff8f9f-902a-44c0-be85-013aceddcd96" containerName="revoke-keys"
Sep 10 11:41:30.929754 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929696    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ff8f9f-902a-44c0-be85-013aceddcd96" containerName="revoke-keys"
Sep 10 11:41:30.929754 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.929706    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="76ac628c-4c68-480b-8518-6b1446cd7329" containerName="payload-processing"
Sep 10 11:41:30.932806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.932789    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:30.935393 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.935361    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:41:30.935508 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.935461    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:41:30.943815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:30.943794    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"]
Sep 10 11:41:31.042142 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.042108    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf\" (UID: \"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:31.142568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.142532    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf\" (UID: \"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:31.143150 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.143131    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf\" (UID: \"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:31.243451 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.243371    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:31.513784 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.513707    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"]
Sep 10 11:41:31.518140 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:41:31.518114    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424ce0ce_ae23_4b01_a82f_3f1dcb9cadc7.slice/crio-e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6 WatchSource:0}: Error finding container e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6: Status 404 returned error can't find the container with id e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6
Sep 10 11:41:31.840468 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.840440    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf" event={"ID":"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7","Type":"ContainerStarted","Data":"1a919e409f0984803cd61cb5a9274ad4df0aba31a0670af27f183f9a2e75c49b"}
Sep 10 11:41:31.840588 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.840485    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf" event={"ID":"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7","Type":"ContainerStarted","Data":"e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6"}
Sep 10 11:41:31.858634 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:31.858593    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf" podStartSLOduration=1.8585781780000001 podStartE2EDuration="1.858578178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:41:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:41:31.857502128 +0000 UTC m=+1626.661569475" watchObservedRunningTime="2026-09-10 11:41:31.858578178 +0000 UTC m=+1626.662645525"
Sep 10 11:41:32.847572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:32.847538    2585 generic.go:356] "Generic (PLEG): container finished" podID="424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" containerID="1a919e409f0984803cd61cb5a9274ad4df0aba31a0670af27f183f9a2e75c49b" exitCode=0
Sep 10 11:41:32.848009 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:32.847630    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf" event={"ID":"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7","Type":"ContainerDied","Data":"1a919e409f0984803cd61cb5a9274ad4df0aba31a0670af27f183f9a2e75c49b"}
Sep 10 11:41:33.998897 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:33.998870    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:34.069422 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.069396    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca\") pod \"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7\" (UID: \"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7\") "
Sep 10 11:41:34.069748 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.069727    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" (UID: "424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:41:34.170750 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.170721    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:34.859250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.859220    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"
Sep 10 11:41:34.859250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.859246    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf" event={"ID":"424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7","Type":"ContainerDied","Data":"e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6"}
Sep 10 11:41:34.859527 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:34.859286    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e456fdcfbfc1fd60eb85918198bac610121645182db57f93b90292607b44a4d6"
Sep 10 11:41:43.887474 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:43.887439    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:41:46.543122 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.543090    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:41:46.543609 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.543330    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" podUID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" containerName="payload-processing" containerID="cri-o://3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca" gracePeriod=30
Sep 10 11:41:46.548547 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.548389    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:41:46.548664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.548617    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" podUID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" containerName="payload-pre-processing" containerID="cri-o://f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac" gracePeriod=30
Sep 10 11:41:46.737028 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.736998    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"]
Sep 10 11:41:46.743587 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.743555    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3f4d4b-wsxtf"]
Sep 10 11:41:46.831932 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.831897    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:41:46.834751 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.834732    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:41:46.922081 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.922043    2585 generic.go:356] "Generic (PLEG): container finished" podID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" containerID="3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca" exitCode=0
Sep 10 11:41:46.922250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.922086    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"
Sep 10 11:41:46.922250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.922134    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" event={"ID":"fbf1f47e-8e01-4c3a-998e-1324819e17ca","Type":"ContainerDied","Data":"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"}
Sep 10 11:41:46.922250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.922191    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6" event={"ID":"fbf1f47e-8e01-4c3a-998e-1324819e17ca","Type":"ContainerDied","Data":"0ceb1012c790a8fda57ca4200bfc5c9b6333f041fe5423217450547771f4d6dc"}
Sep 10 11:41:46.922250 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.922220    2585 scope.go:117] "RemoveContainer" containerID="3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"
Sep 10 11:41:46.923699 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.923677    2585 generic.go:356] "Generic (PLEG): container finished" podID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" containerID="f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac" exitCode=0
Sep 10 11:41:46.923805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.923732    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" event={"ID":"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff","Type":"ContainerDied","Data":"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"}
Sep 10 11:41:46.923805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.923762    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"
Sep 10 11:41:46.923805 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.923778    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2" event={"ID":"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff","Type":"ContainerDied","Data":"c080d268de3c97c69de5f72faa7d5faec4f1f81d2446c0ca7c3bed456a4a5b8a"}
Sep 10 11:41:46.932401 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.932378    2585 scope.go:117] "RemoveContainer" containerID="3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"
Sep 10 11:41:46.932673 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:41:46.932653    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca\": container with ID starting with 3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca not found: ID does not exist" containerID="3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"
Sep 10 11:41:46.932751 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.932679    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca"} err="failed to get container status \"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca\": rpc error: code = NotFound desc = could not find container \"3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca\": container with ID starting with 3896db9e96ceedf84e288e6a8b66372ffc154b30b86f946d543dee172ea5f1ca not found: ID does not exist"
Sep 10 11:41:46.932751 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.932695    2585 scope.go:117] "RemoveContainer" containerID="f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"
Sep 10 11:41:46.940700 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.940679    2585 scope.go:117] "RemoveContainer" containerID="f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"
Sep 10 11:41:46.940942 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:41:46.940925    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac\": container with ID starting with f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac not found: ID does not exist" containerID="f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"
Sep 10 11:41:46.940988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.940948    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac"} err="failed to get container status \"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac\": rpc error: code = NotFound desc = could not find container \"f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac\": container with ID starting with f2447297a51bb5231d6187ca380a6ee90b3fbe00d3bf15673bda2b637da6e6ac not found: ID does not exist"
Sep 10 11:41:46.984262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984225    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wqvj\" (UniqueName: \"kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj\") pod \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") "
Sep 10 11:41:46.984370 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984284    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wjg9\" (UniqueName: \"kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9\") pod \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") "
Sep 10 11:41:46.984433 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984415    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume\") pod \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\" (UID: \"d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff\") "
Sep 10 11:41:46.984490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984453    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume\") pod \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\" (UID: \"fbf1f47e-8e01-4c3a-998e-1324819e17ca\") "
Sep 10 11:41:46.984808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984780    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fbf1f47e-8e01-4c3a-998e-1324819e17ca" (UID: "fbf1f47e-8e01-4c3a-998e-1324819e17ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:41:46.984909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.984780    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" (UID: "d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:41:46.986265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.986234    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj" (OuterVolumeSpecName: "kube-api-access-8wqvj") pod "d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" (UID: "d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff"). InnerVolumeSpecName "kube-api-access-8wqvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:41:46.986384 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:46.986273    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9" (OuterVolumeSpecName: "kube-api-access-5wjg9") pod "fbf1f47e-8e01-4c3a-998e-1324819e17ca" (UID: "fbf1f47e-8e01-4c3a-998e-1324819e17ca"). InnerVolumeSpecName "kube-api-access-5wjg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:41:47.086079 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.086014    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wqvj\" (UniqueName: \"kubernetes.io/projected/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-kube-api-access-8wqvj\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:47.086079 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.086038    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wjg9\" (UniqueName: \"kubernetes.io/projected/fbf1f47e-8e01-4c3a-998e-1324819e17ca-kube-api-access-5wjg9\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:47.086079 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.086048    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:47.086079 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.086056    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf1f47e-8e01-4c3a-998e-1324819e17ca-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:41:47.260881 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.260851    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:41:47.267257 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.267232    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3f4d4b-5fd694f6db-m8kg2"]
Sep 10 11:41:47.290185 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.290153    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:41:47.312831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.312804    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3f4d4b-84d66d4c8c-bt9t6"]
Sep 10 11:41:47.663723 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.663690    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" path="/var/lib/kubelet/pods/424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7/volumes"
Sep 10 11:41:47.664094 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.664068    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" path="/var/lib/kubelet/pods/d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff/volumes"
Sep 10 11:41:47.664395 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:47.664381    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" path="/var/lib/kubelet/pods/fbf1f47e-8e01-4c3a-998e-1324819e17ca/volumes"
Sep 10 11:41:52.127660 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:52.127624    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:41:56.964573 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:41:56.964537    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:42:07.147579 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.147528    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" podUID="09c14362-86c6-490b-9bcb-acc28c3c22af" containerName="main" containerID="cri-o://58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75" gracePeriod=45
Sep 10 11:42:07.376884 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.376860    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:42:07.453277 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453252    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453283    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453332    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj7dh\" (UniqueName: \"kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453391    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453499    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache" (OuterVolumeSpecName: "model-cache") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:42:07.453753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453559    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453644    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir\") pod \"09c14362-86c6-490b-9bcb-acc28c3c22af\" (UID: \"09c14362-86c6-490b-9bcb-acc28c3c22af\") "
Sep 10 11:42:07.453753 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453677    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home" (OuterVolumeSpecName: "home") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:42:07.454041 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453955    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:07.454041 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453973    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:07.454041 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.453982    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:42:07.455675 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.455654    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm" (OuterVolumeSpecName: "dshm") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:42:07.455799 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.455779    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:42:07.455889 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.455871    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh" (OuterVolumeSpecName: "kube-api-access-vj7dh") pod "09c14362-86c6-490b-9bcb-acc28c3c22af" (UID: "09c14362-86c6-490b-9bcb-acc28c3c22af"). InnerVolumeSpecName "kube-api-access-vj7dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:42:07.555003 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.554968    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:07.555003 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.554994    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c14362-86c6-490b-9bcb-acc28c3c22af-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:07.555003 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.555006    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c14362-86c6-490b-9bcb-acc28c3c22af-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:07.555232 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:07.555017    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj7dh\" (UniqueName: \"kubernetes.io/projected/09c14362-86c6-490b-9bcb-acc28c3c22af-kube-api-access-vj7dh\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:08.038719 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.038682    2585 generic.go:356] "Generic (PLEG): container finished" podID="09c14362-86c6-490b-9bcb-acc28c3c22af" containerID="58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75" exitCode=0
Sep 10 11:42:08.039016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.038729    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"
Sep 10 11:42:08.039016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.038762    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" event={"ID":"09c14362-86c6-490b-9bcb-acc28c3c22af","Type":"ContainerDied","Data":"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"}
Sep 10 11:42:08.039016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.038798    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j" event={"ID":"09c14362-86c6-490b-9bcb-acc28c3c22af","Type":"ContainerDied","Data":"67a94ec76075e2cbbfe88bb5baa7f920e63b8d2b6c405850dbde275788f91d57"}
Sep 10 11:42:08.039016 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.038814    2585 scope.go:117] "RemoveContainer" containerID="58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"
Sep 10 11:42:08.047358 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.047339    2585 scope.go:117] "RemoveContainer" containerID="58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"
Sep 10 11:42:08.047636 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:42:08.047620    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75\": container with ID starting with 58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75 not found: ID does not exist" containerID="58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"
Sep 10 11:42:08.047686 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.047644    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75"} err="failed to get container status \"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75\": rpc error: code = NotFound desc = could not find container \"58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75\": container with ID starting with 58a171c35cc0290aa3c6a1e6b2383d5dc9f8e7e2396dcae08e0fc714664a9a75 not found: ID does not exist"
Sep 10 11:42:08.072441 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.072418    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:42:08.077915 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:08.077895    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-3f4d4b/ipp-route-3f4d4b-kserve-7c857cf5ff-n856j"]
Sep 10 11:42:09.663072 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:09.663029    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09c14362-86c6-490b-9bcb-acc28c3c22af" path="/var/lib/kubelet/pods/09c14362-86c6-490b-9bcb-acc28c3c22af/volumes"
Sep 10 11:42:36.516067 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516024    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"]
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516424    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" containerName="payload-processing"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516436    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" containerName="payload-processing"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516447    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c14362-86c6-490b-9bcb-acc28c3c22af" containerName="main"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516453    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c14362-86c6-490b-9bcb-acc28c3c22af" containerName="main"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516465    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" containerName="revoke-keys"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516471    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" containerName="revoke-keys"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516485    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" containerName="payload-pre-processing"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516491    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" containerName="payload-pre-processing"
Sep 10 11:42:36.516545 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516550    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c14362-86c6-490b-9bcb-acc28c3c22af" containerName="main"
Sep 10 11:42:36.516836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516559    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="d49dd90f-0bd9-45cb-b9bb-9dad3bab31ff" containerName="payload-pre-processing"
Sep 10 11:42:36.516836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516565    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf1f47e-8e01-4c3a-998e-1324819e17ca" containerName="payload-processing"
Sep 10 11:42:36.516836 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.516572    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="424ce0ce-ae23-4b01-a82f-3f1dcb9cadc7" containerName="revoke-keys"
Sep 10 11:42:36.519680 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.519661    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:36.522168 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.522143    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:42:36.523097 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.523074    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:42:36.529699 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.529073    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"]
Sep 10 11:42:36.617985 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.617946    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q\" (UID: \"fec39b05-5d4e-4cb8-a506-d1af2029dc59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:36.718937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.718894    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q\" (UID: \"fec39b05-5d4e-4cb8-a506-d1af2029dc59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:36.719588 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.719566    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q\" (UID: \"fec39b05-5d4e-4cb8-a506-d1af2029dc59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:36.834487 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:36.834394    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:37.107195 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:37.107167    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"]
Sep 10 11:42:37.109112 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:42:37.109084    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec39b05_5d4e_4cb8_a506_d1af2029dc59.slice/crio-f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b WatchSource:0}: Error finding container f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b: Status 404 returned error can't find the container with id f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b
Sep 10 11:42:37.190000 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:37.189967    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q" event={"ID":"fec39b05-5d4e-4cb8-a506-d1af2029dc59","Type":"ContainerStarted","Data":"f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b"}
Sep 10 11:42:38.203687 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:38.203645    2585 generic.go:356] "Generic (PLEG): container finished" podID="fec39b05-5d4e-4cb8-a506-d1af2029dc59" containerID="5a671ccba34fd3557082e5d472dcf00204a2391b5afbb42781b79f704d59e45d" exitCode=0
Sep 10 11:42:38.204085 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:38.203736    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q" event={"ID":"fec39b05-5d4e-4cb8-a506-d1af2029dc59","Type":"ContainerDied","Data":"5a671ccba34fd3557082e5d472dcf00204a2391b5afbb42781b79f704d59e45d"}
Sep 10 11:42:39.339810 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:39.339783    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:39.441172 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:39.441128    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca\") pod \"fec39b05-5d4e-4cb8-a506-d1af2029dc59\" (UID: \"fec39b05-5d4e-4cb8-a506-d1af2029dc59\") "
Sep 10 11:42:39.441572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:39.441552    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fec39b05-5d4e-4cb8-a506-d1af2029dc59" (UID: "fec39b05-5d4e-4cb8-a506-d1af2029dc59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:42:39.542950 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:39.542857    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fec39b05-5d4e-4cb8-a506-d1af2029dc59-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:40.215365 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:40.215316    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q" event={"ID":"fec39b05-5d4e-4cb8-a506-d1af2029dc59","Type":"ContainerDied","Data":"f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b"}
Sep 10 11:42:40.215550 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:40.215389    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1b69bf814c3b7e46228f192263369ecd65d0bb8e698ffde4c566cdc19fd230b"
Sep 10 11:42:40.215550 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:40.215336    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"
Sep 10 11:42:47.291639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.291600    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:42:47.292245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.291843    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerName="payload-pre-processing" containerID="cri-o://af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1" gracePeriod=30
Sep 10 11:42:47.294946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.294915    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:42:47.294946 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.295149    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerName="payload-processing" containerID="cri-o://0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0" gracePeriod=30
Sep 10 11:42:47.300571 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.300542    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.80:9004: connect: connection refused"
Sep 10 11:42:47.300837 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.300803    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.79:9004: connect: connection refused"
Sep 10 11:42:47.473420 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.473385    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"]
Sep 10 11:42:47.477353 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.477318    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-420024-ssb8q"]
Sep 10 11:42:47.573515 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.573489    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:42:47.576481 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.576457    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:42:47.617372 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.617336    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume\") pod \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") "
Sep 10 11:42:47.617580 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.617395    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmtsp\" (UniqueName: \"kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp\") pod \"cc07336e-3ab0-4b3b-a537-755baed2878c\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") "
Sep 10 11:42:47.617580 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.617480    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-787nx\" (UniqueName: \"kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx\") pod \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\" (UID: \"1d4df18f-9bcb-4ccf-9407-d493845b5d53\") "
Sep 10 11:42:47.617580 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.617562    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume\") pod \"cc07336e-3ab0-4b3b-a537-755baed2878c\" (UID: \"cc07336e-3ab0-4b3b-a537-755baed2878c\") "
Sep 10 11:42:47.617819 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.617791    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d4df18f-9bcb-4ccf-9407-d493845b5d53" (UID: "1d4df18f-9bcb-4ccf-9407-d493845b5d53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:42:47.618105 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.618081    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cc07336e-3ab0-4b3b-a537-755baed2878c" (UID: "cc07336e-3ab0-4b3b-a537-755baed2878c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:42:47.619789 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.619764    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp" (OuterVolumeSpecName: "kube-api-access-jmtsp") pod "cc07336e-3ab0-4b3b-a537-755baed2878c" (UID: "cc07336e-3ab0-4b3b-a537-755baed2878c"). InnerVolumeSpecName "kube-api-access-jmtsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:42:47.619789 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.619776    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx" (OuterVolumeSpecName: "kube-api-access-787nx") pod "1d4df18f-9bcb-4ccf-9407-d493845b5d53" (UID: "1d4df18f-9bcb-4ccf-9407-d493845b5d53"). InnerVolumeSpecName "kube-api-access-787nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:42:47.665490 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.665448    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fec39b05-5d4e-4cb8-a506-d1af2029dc59" path="/var/lib/kubelet/pods/fec39b05-5d4e-4cb8-a506-d1af2029dc59/volumes"
Sep 10 11:42:47.718664 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.718622    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-787nx\" (UniqueName: \"kubernetes.io/projected/1d4df18f-9bcb-4ccf-9407-d493845b5d53-kube-api-access-787nx\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:47.718926 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.718899    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cc07336e-3ab0-4b3b-a537-755baed2878c-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:47.718926 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.718930    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d4df18f-9bcb-4ccf-9407-d493845b5d53-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:47.719094 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:47.718944    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmtsp\" (UniqueName: \"kubernetes.io/projected/cc07336e-3ab0-4b3b-a537-755baed2878c-kube-api-access-jmtsp\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:48.257434 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.257344    2585 generic.go:356] "Generic (PLEG): container finished" podID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerID="0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0" exitCode=0
Sep 10 11:42:48.257434 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.257391    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"
Sep 10 11:42:48.257677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.257429    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" event={"ID":"cc07336e-3ab0-4b3b-a537-755baed2878c","Type":"ContainerDied","Data":"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"}
Sep 10 11:42:48.257677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.257486    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk" event={"ID":"cc07336e-3ab0-4b3b-a537-755baed2878c","Type":"ContainerDied","Data":"f1a03564792677c0361ecc278d41b2f83f4b45a89cb47a18f1cc649b4989aa63"}
Sep 10 11:42:48.257677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.257520    2585 scope.go:117] "RemoveContainer" containerID="0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"
Sep 10 11:42:48.258887 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.258866    2585 generic.go:356] "Generic (PLEG): container finished" podID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerID="af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1" exitCode=0
Sep 10 11:42:48.259018 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.258902    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" event={"ID":"1d4df18f-9bcb-4ccf-9407-d493845b5d53","Type":"ContainerDied","Data":"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"}
Sep 10 11:42:48.259018 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.258921    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"
Sep 10 11:42:48.259018 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.258951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h" event={"ID":"1d4df18f-9bcb-4ccf-9407-d493845b5d53","Type":"ContainerDied","Data":"267acc212cc4b007837a03ead4da69c2bea61c9825a3ea5efd5609fa760e1316"}
Sep 10 11:42:48.266473 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.266455    2585 scope.go:117] "RemoveContainer" containerID="0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"
Sep 10 11:42:48.266730 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:42:48.266711    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0\": container with ID starting with 0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0 not found: ID does not exist" containerID="0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"
Sep 10 11:42:48.266780 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.266739    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0"} err="failed to get container status \"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0\": rpc error: code = NotFound desc = could not find container \"0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0\": container with ID starting with 0e4655ba5ed46a6e124ca3b665f8bea540ec41f4ffc4013b05c3f3b67f3b5cd0 not found: ID does not exist"
Sep 10 11:42:48.266780 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.266756    2585 scope.go:117] "RemoveContainer" containerID="af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"
Sep 10 11:42:48.274668 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.274646    2585 scope.go:117] "RemoveContainer" containerID="af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"
Sep 10 11:42:48.275067 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:42:48.275039    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1\": container with ID starting with af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1 not found: ID does not exist" containerID="af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"
Sep 10 11:42:48.275176 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.275077    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1"} err="failed to get container status \"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1\": rpc error: code = NotFound desc = could not find container \"af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1\": container with ID starting with af87ce8d4d1e6836ff1d8c0bf1da6b17f1b43feffe89163f3e21b7ffb92e46b1 not found: ID does not exist"
Sep 10 11:42:48.277191 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.277169    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:42:48.281346 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.281320    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-420024-5cc769c9d-7nv2h"]
Sep 10 11:42:48.292835 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.292811    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:42:48.294532 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:48.294514    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-420024-6766fb9cf5-qxjdk"]
Sep 10 11:42:49.663572 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:49.663538    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" path="/var/lib/kubelet/pods/1d4df18f-9bcb-4ccf-9407-d493845b5d53/volumes"
Sep 10 11:42:49.663941 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:49.663893    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" path="/var/lib/kubelet/pods/cc07336e-3ab0-4b3b-a537-755baed2878c/volumes"
Sep 10 11:42:50.480756 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:50.480721    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:42:55.407922 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.407872    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"]
Sep 10 11:42:55.408345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408329    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerName="payload-pre-processing"
Sep 10 11:42:55.408345 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408346    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerName="payload-pre-processing"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408353    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec39b05-5d4e-4cb8-a506-d1af2029dc59" containerName="revoke-keys"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408359    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec39b05-5d4e-4cb8-a506-d1af2029dc59" containerName="revoke-keys"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408369    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerName="payload-processing"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408374    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerName="payload-processing"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408428    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc07336e-3ab0-4b3b-a537-755baed2878c" containerName="payload-processing"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408437    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4df18f-9bcb-4ccf-9407-d493845b5d53" containerName="payload-pre-processing"
Sep 10 11:42:55.408449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.408444    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec39b05-5d4e-4cb8-a506-d1af2029dc59" containerName="revoke-keys"
Sep 10 11:42:55.412917 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.412898    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:55.415211 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.415187    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:42:55.416104 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.416087    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:42:55.433831 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.433800    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"]
Sep 10 11:42:55.485901 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.485856    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq\" (UID: \"0b26c6ea-51a6-4888-ac43-89c280b81774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:55.586981 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.586948    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq\" (UID: \"0b26c6ea-51a6-4888-ac43-89c280b81774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:55.587563 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.587544    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq\" (UID: \"0b26c6ea-51a6-4888-ac43-89c280b81774\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:55.727828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:55.727790    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:56.004894 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:56.004710    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"]
Sep 10 11:42:56.007768 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:42:56.007743    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b26c6ea_51a6_4888_ac43_89c280b81774.slice/crio-3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4 WatchSource:0}: Error finding container 3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4: Status 404 returned error can't find the container with id 3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4
Sep 10 11:42:56.301975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:56.301901    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq" event={"ID":"0b26c6ea-51a6-4888-ac43-89c280b81774","Type":"ContainerStarted","Data":"2020abf5b955e18e23b5b7f92560c5d9d7b945f27002a50bdd1d53d2e9959114"}
Sep 10 11:42:56.301975 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:56.301956    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq" event={"ID":"0b26c6ea-51a6-4888-ac43-89c280b81774","Type":"ContainerStarted","Data":"3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4"}
Sep 10 11:42:56.318626 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:56.318574    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq" podStartSLOduration=1.318554993 podStartE2EDuration="1.318554993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:42:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:42:56.318068318 +0000 UTC m=+1711.122135668" watchObservedRunningTime="2026-09-10 11:42:56.318554993 +0000 UTC m=+1711.122622341"
Sep 10 11:42:57.309840 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:57.309809    2585 generic.go:356] "Generic (PLEG): container finished" podID="0b26c6ea-51a6-4888-ac43-89c280b81774" containerID="2020abf5b955e18e23b5b7f92560c5d9d7b945f27002a50bdd1d53d2e9959114" exitCode=0
Sep 10 11:42:57.310267 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:57.309896    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq" event={"ID":"0b26c6ea-51a6-4888-ac43-89c280b81774","Type":"ContainerDied","Data":"2020abf5b955e18e23b5b7f92560c5d9d7b945f27002a50bdd1d53d2e9959114"}
Sep 10 11:42:58.442291 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:58.442267    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:42:58.513529 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:58.513500    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca\") pod \"0b26c6ea-51a6-4888-ac43-89c280b81774\" (UID: \"0b26c6ea-51a6-4888-ac43-89c280b81774\") "
Sep 10 11:42:58.513845 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:58.513821    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b26c6ea-51a6-4888-ac43-89c280b81774" (UID: "0b26c6ea-51a6-4888-ac43-89c280b81774"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:42:58.615002 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:58.614932    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b26c6ea-51a6-4888-ac43-89c280b81774-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:42:59.321853 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:59.321822    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq" event={"ID":"0b26c6ea-51a6-4888-ac43-89c280b81774","Type":"ContainerDied","Data":"3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4"}
Sep 10 11:42:59.322031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:59.321874    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3897abf94e4cdd062e0c4d6d4af7ced021747cecef53b506d1bff09af813b0d4"
Sep 10 11:42:59.322031 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:42:59.321832    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"
Sep 10 11:43:06.435767 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.435722    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:43:06.436235 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.435962    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" podUID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" containerName="payload-pre-processing" containerID="cri-o://1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f" gracePeriod=30
Sep 10 11:43:06.637074 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.637034    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"]
Sep 10 11:43:06.640940 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.640909    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-5fde2a-lqnjq"]
Sep 10 11:43:06.696235 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.696169    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:43:06.789560 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.789509    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcllv\" (UniqueName: \"kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv\") pod \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") "
Sep 10 11:43:06.789732 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.789715    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume\") pod \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\" (UID: \"a25ac598-84c8-47c9-9fb0-1d5863c99abb\") "
Sep 10 11:43:06.790128 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.790087    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a25ac598-84c8-47c9-9fb0-1d5863c99abb" (UID: "a25ac598-84c8-47c9-9fb0-1d5863c99abb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:43:06.791716 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.791693    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv" (OuterVolumeSpecName: "kube-api-access-bcllv") pod "a25ac598-84c8-47c9-9fb0-1d5863c99abb" (UID: "a25ac598-84c8-47c9-9fb0-1d5863c99abb"). InnerVolumeSpecName "kube-api-access-bcllv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:43:06.890988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.890948    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a25ac598-84c8-47c9-9fb0-1d5863c99abb-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:43:06.890988 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:06.890982    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcllv\" (UniqueName: \"kubernetes.io/projected/a25ac598-84c8-47c9-9fb0-1d5863c99abb-kube-api-access-bcllv\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:43:07.370245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.370206    2585 generic.go:356] "Generic (PLEG): container finished" podID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" containerID="1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f" exitCode=0
Sep 10 11:43:07.370245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.370246    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"
Sep 10 11:43:07.370493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.370291    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" event={"ID":"a25ac598-84c8-47c9-9fb0-1d5863c99abb","Type":"ContainerDied","Data":"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"}
Sep 10 11:43:07.370493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.370383    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr" event={"ID":"a25ac598-84c8-47c9-9fb0-1d5863c99abb","Type":"ContainerDied","Data":"0e9989284b9f319b802b04c785121a38f93ff03ae544ff68990efbc651022e84"}
Sep 10 11:43:07.370493 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.370406    2585 scope.go:117] "RemoveContainer" containerID="1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"
Sep 10 11:43:07.379466 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.379448    2585 scope.go:117] "RemoveContainer" containerID="1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"
Sep 10 11:43:07.379752 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:43:07.379730    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f\": container with ID starting with 1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f not found: ID does not exist" containerID="1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"
Sep 10 11:43:07.379815 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.379762    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f"} err="failed to get container status \"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f\": rpc error: code = NotFound desc = could not find container \"1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f\": container with ID starting with 1afc7a2016bc2d3c365155cd2d119271595c4142fdc2543528e01cdc68bbe09f not found: ID does not exist"
Sep 10 11:43:07.395110 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.395075    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:43:07.403126 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.403094    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-5fde2a-5998598766-4khqr"]
Sep 10 11:43:07.663910 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.663877    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b26c6ea-51a6-4888-ac43-89c280b81774" path="/var/lib/kubelet/pods/0b26c6ea-51a6-4888-ac43-89c280b81774/volumes"
Sep 10 11:43:07.664286 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:07.664244    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" path="/var/lib/kubelet/pods/a25ac598-84c8-47c9-9fb0-1d5863c99abb/volumes"
Sep 10 11:43:09.368011 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:09.367969    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:43:14.136646 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.136607    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"]
Sep 10 11:43:14.137012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.136959    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" containerName="payload-pre-processing"
Sep 10 11:43:14.137012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.136970    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" containerName="payload-pre-processing"
Sep 10 11:43:14.137012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.136979    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b26c6ea-51a6-4888-ac43-89c280b81774" containerName="revoke-keys"
Sep 10 11:43:14.137012 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.136985    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b26c6ea-51a6-4888-ac43-89c280b81774" containerName="revoke-keys"
Sep 10 11:43:14.137156 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.137053    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a25ac598-84c8-47c9-9fb0-1d5863c99abb" containerName="payload-pre-processing"
Sep 10 11:43:14.137156 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.137062    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b26c6ea-51a6-4888-ac43-89c280b81774" containerName="revoke-keys"
Sep 10 11:43:14.141527 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.141508    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:14.143742 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.143716    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-99z58\""
Sep 10 11:43:14.143972 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.143954    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:43:14.148146 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.148120    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"]
Sep 10 11:43:14.255567 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.255525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf\" (UID: \"f39bf42d-42f5-4cb0-b929-f5d77952a1c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:14.356959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.356918    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf\" (UID: \"f39bf42d-42f5-4cb0-b929-f5d77952a1c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:14.357595 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.357577    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf\" (UID: \"f39bf42d-42f5-4cb0-b929-f5d77952a1c6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:14.453104 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.453066    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:14.725794 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:14.725704    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"]
Sep 10 11:43:14.729909 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:43:14.729881    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39bf42d_42f5_4cb0_b929_f5d77952a1c6.slice/crio-7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2 WatchSource:0}: Error finding container 7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2: Status 404 returned error can't find the container with id 7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2
Sep 10 11:43:15.416104 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:15.416073    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf" event={"ID":"f39bf42d-42f5-4cb0-b929-f5d77952a1c6","Type":"ContainerStarted","Data":"ecb593967683e359873032176fd6e70b06c51ca3a06f949c94a043c03f55ca7d"}
Sep 10 11:43:15.416467 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:15.416129    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf" event={"ID":"f39bf42d-42f5-4cb0-b929-f5d77952a1c6","Type":"ContainerStarted","Data":"7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2"}
Sep 10 11:43:15.433169 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:15.433115    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf" podStartSLOduration=1.433099215 podStartE2EDuration="1.433099215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:43:15.431600606 +0000 UTC m=+1730.235667954" watchObservedRunningTime="2026-09-10 11:43:15.433099215 +0000 UTC m=+1730.237166572"
Sep 10 11:43:16.423096 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:16.423061    2585 generic.go:356] "Generic (PLEG): container finished" podID="f39bf42d-42f5-4cb0-b929-f5d77952a1c6" containerID="ecb593967683e359873032176fd6e70b06c51ca3a06f949c94a043c03f55ca7d" exitCode=0
Sep 10 11:43:16.423637 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:16.423151    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf" event={"ID":"f39bf42d-42f5-4cb0-b929-f5d77952a1c6","Type":"ContainerDied","Data":"ecb593967683e359873032176fd6e70b06c51ca3a06f949c94a043c03f55ca7d"}
Sep 10 11:43:17.562974 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:17.562945    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:17.689648 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:17.689544    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca\") pod \"f39bf42d-42f5-4cb0-b929-f5d77952a1c6\" (UID: \"f39bf42d-42f5-4cb0-b929-f5d77952a1c6\") "
Sep 10 11:43:17.689923 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:17.689898    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f39bf42d-42f5-4cb0-b929-f5d77952a1c6" (UID: "f39bf42d-42f5-4cb0-b929-f5d77952a1c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:43:17.790533 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:17.790493    2585 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f39bf42d-42f5-4cb0-b929-f5d77952a1c6-maas-api-service-ca\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:43:18.435007 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:18.434974    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"
Sep 10 11:43:18.435294 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:18.435019    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf" event={"ID":"f39bf42d-42f5-4cb0-b929-f5d77952a1c6","Type":"ContainerDied","Data":"7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2"}
Sep 10 11:43:18.435294 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:18.435075    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fae72e231f0090e6a67bd8ae81fb9ca6227a69e622c6af7f68815006c1c63a2"
Sep 10 11:43:24.735474 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:24.735437    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:43:24.738822 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:24.738779    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" podUID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" containerName="payload-processing" containerID="cri-o://01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55" gracePeriod=30
Sep 10 11:43:24.947579 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:24.947534    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"]
Sep 10 11:43:24.949418 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:24.949386    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-13c392-2fnxf"]
Sep 10 11:43:25.002246 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.002219    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:43:25.155847 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.155814    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbhvg\" (UniqueName: \"kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg\") pod \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") "
Sep 10 11:43:25.156027 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.155883    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume\") pod \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\" (UID: \"a7a6f614-c94f-4d31-8b03-c2ba51d215aa\") "
Sep 10 11:43:25.156238 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.156215    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7a6f614-c94f-4d31-8b03-c2ba51d215aa" (UID: "a7a6f614-c94f-4d31-8b03-c2ba51d215aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 11:43:25.157989 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.157966    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg" (OuterVolumeSpecName: "kube-api-access-sbhvg") pod "a7a6f614-c94f-4d31-8b03-c2ba51d215aa" (UID: "a7a6f614-c94f-4d31-8b03-c2ba51d215aa"). InnerVolumeSpecName "kube-api-access-sbhvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:43:25.257050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.256933    2585 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-plugins-config-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:43:25.257050 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.256985    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbhvg\" (UniqueName: \"kubernetes.io/projected/a7a6f614-c94f-4d31-8b03-c2ba51d215aa-kube-api-access-sbhvg\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:43:25.470551 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.470517    2585 generic.go:356] "Generic (PLEG): container finished" podID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" containerID="01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55" exitCode=0
Sep 10 11:43:25.470722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.470561    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"
Sep 10 11:43:25.470722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.470603    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" event={"ID":"a7a6f614-c94f-4d31-8b03-c2ba51d215aa","Type":"ContainerDied","Data":"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"}
Sep 10 11:43:25.470722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.470658    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58" event={"ID":"a7a6f614-c94f-4d31-8b03-c2ba51d215aa","Type":"ContainerDied","Data":"9a5073646649287ad0791dde9c0fef3005a4e2d834ef9f88e2f56c07dd5a05eb"}
Sep 10 11:43:25.470722 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.470687    2585 scope.go:117] "RemoveContainer" containerID="01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"
Sep 10 11:43:25.480322 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.480282    2585 scope.go:117] "RemoveContainer" containerID="01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"
Sep 10 11:43:25.480568 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:43:25.480551    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55\": container with ID starting with 01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55 not found: ID does not exist" containerID="01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"
Sep 10 11:43:25.480622 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.480575    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55"} err="failed to get container status \"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55\": rpc error: code = NotFound desc = could not find container \"01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55\": container with ID starting with 01cc82d73d9650a3b46971f999e1470ed181b9f6f98882c82d05b3094dfe9e55 not found: ID does not exist"
Sep 10 11:43:25.507535 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.507454    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:43:25.521656 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.521628    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-13c392-64fcdbdcdc-48v58"]
Sep 10 11:43:25.663667 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.663632    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" path="/var/lib/kubelet/pods/a7a6f614-c94f-4d31-8b03-c2ba51d215aa/volumes"
Sep 10 11:43:25.664060 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:25.664046    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f39bf42d-42f5-4cb0-b929-f5d77952a1c6" path="/var/lib/kubelet/pods/f39bf42d-42f5-4cb0-b929-f5d77952a1c6/volumes"
Sep 10 11:43:26.017310 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:26.017263    2585 scope.go:117] "RemoveContainer" containerID="fc423d286921a7dedbaaa68733508ecd579c78e4e18ec77efeaddee1982f8cd6"
Sep 10 11:43:26.028265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:26.028242    2585 scope.go:117] "RemoveContainer" containerID="70845da46435d65a51dd25d7cd8d6cbaa5de59732c0a154cb03599d04bae498d"
Sep 10 11:43:27.371393 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:27.371355    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:43:37.368603 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:43:37.368574    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:44:25.745074 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:44:25.745043    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:44:25.749377 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:44:25.749355    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:44:56.965232 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:44:56.965199    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:45:22.479795 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:45:22.479756    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:45:26.073032 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:45:26.072997    2585 scope.go:117] "RemoveContainer" containerID="49ea609d5ed7a274dc03b161458bbe87824aa6b3bff411b6af4c51ff8a28f616"
Sep 10 11:45:30.785419 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:45:30.785382    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:45:39.477278 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:45:39.477241    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:45:56.469357 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:45:56.469264    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:46:03.499844 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:03.499806    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:46:06.486547 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:06.486510    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:46:06.487010 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:06.486741    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" podUID="11c32ebb-5818-4132-bac4-f9e59f6376bc" containerName="manager" containerID="cri-o://cb2b5bb61315638bd7214b73aae8784f174249212d24da00c684aa2d54e3c986" gracePeriod=10
Sep 10 11:46:07.280821 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.280788    2585 generic.go:356] "Generic (PLEG): container finished" podID="11c32ebb-5818-4132-bac4-f9e59f6376bc" containerID="cb2b5bb61315638bd7214b73aae8784f174249212d24da00c684aa2d54e3c986" exitCode=0
Sep 10 11:46:07.280966 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.280857    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" event={"ID":"11c32ebb-5818-4132-bac4-f9e59f6376bc","Type":"ContainerDied","Data":"cb2b5bb61315638bd7214b73aae8784f174249212d24da00c684aa2d54e3c986"}
Sep 10 11:46:07.333888 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.333860    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:46:07.387260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.387173    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume\") pod \"11c32ebb-5818-4132-bac4-f9e59f6376bc\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") "
Sep 10 11:46:07.387260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.387218    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skr6m\" (UniqueName: \"kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m\") pod \"11c32ebb-5818-4132-bac4-f9e59f6376bc\" (UID: \"11c32ebb-5818-4132-bac4-f9e59f6376bc\") "
Sep 10 11:46:07.387621 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.387598    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "11c32ebb-5818-4132-bac4-f9e59f6376bc" (UID: "11c32ebb-5818-4132-bac4-f9e59f6376bc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:46:07.389398 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.389368    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m" (OuterVolumeSpecName: "kube-api-access-skr6m") pod "11c32ebb-5818-4132-bac4-f9e59f6376bc" (UID: "11c32ebb-5818-4132-bac4-f9e59f6376bc"). InnerVolumeSpecName "kube-api-access-skr6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:46:07.488892 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.488851    2585 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11c32ebb-5818-4132-bac4-f9e59f6376bc-extensions-socket-volume\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:46:07.488892 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:07.488883    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skr6m\" (UniqueName: \"kubernetes.io/projected/11c32ebb-5818-4132-bac4-f9e59f6376bc-kube-api-access-skr6m\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:46:08.293417 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:08.293381    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72" event={"ID":"11c32ebb-5818-4132-bac4-f9e59f6376bc","Type":"ContainerDied","Data":"ccb5ccadab7cd4352e738d354572323d92ab1b03994b7fed56b73b7c5756c205"}
Sep 10 11:46:08.293591 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:08.293465    2585 scope.go:117] "RemoveContainer" containerID="cb2b5bb61315638bd7214b73aae8784f174249212d24da00c684aa2d54e3c986"
Sep 10 11:46:08.293591 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:08.293397    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"
Sep 10 11:46:08.313208 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:08.313182    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:46:08.323213 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:08.323188    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-k7x72"]
Sep 10 11:46:09.664387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:46:09.664354    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c32ebb-5818-4132-bac4-f9e59f6376bc" path="/var/lib/kubelet/pods/11c32ebb-5818-4132-bac4-f9e59f6376bc/volumes"
Sep 10 11:47:07.223830 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.223797    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"]
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224126    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11c32ebb-5818-4132-bac4-f9e59f6376bc" containerName="manager"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224138    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c32ebb-5818-4132-bac4-f9e59f6376bc" containerName="manager"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224147    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f39bf42d-42f5-4cb0-b929-f5d77952a1c6" containerName="revoke-keys"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224153    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f39bf42d-42f5-4cb0-b929-f5d77952a1c6" containerName="revoke-keys"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224162    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" containerName="payload-processing"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224168    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" containerName="payload-processing"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224231    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="11c32ebb-5818-4132-bac4-f9e59f6376bc" containerName="manager"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224239    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f39bf42d-42f5-4cb0-b929-f5d77952a1c6" containerName="revoke-keys"
Sep 10 11:47:07.224247 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.224245    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a6f614-c94f-4d31-8b03-c2ba51d215aa" containerName="payload-processing"
Sep 10 11:47:07.227311 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.227284    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.229443 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.229424    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ngvwc\""
Sep 10 11:47:07.246578 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.246549    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"]
Sep 10 11:47:07.292670 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.292639    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.292837 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.292752    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz4ws\" (UniqueName: \"kubernetes.io/projected/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-kube-api-access-tz4ws\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.394089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.394046    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz4ws\" (UniqueName: \"kubernetes.io/projected/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-kube-api-access-tz4ws\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.394282 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.394117    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.394528 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.394510    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.404820 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.404802    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz4ws\" (UniqueName: \"kubernetes.io/projected/9f38ad4d-05bc-4cea-a484-d8de6e9fc93d-kube-api-access-tz4ws\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zxcz6\" (UID: \"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.537024 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.536938    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:07.828274 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:07.828195    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"]
Sep 10 11:47:07.831447 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:47:07.831420    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f38ad4d_05bc_4cea_a484_d8de6e9fc93d.slice/crio-f5fa3a8ae9e696cbd517f3ea73bbdaadd11f53ebe8079d1c38ec3f947a04331e WatchSource:0}: Error finding container f5fa3a8ae9e696cbd517f3ea73bbdaadd11f53ebe8079d1c38ec3f947a04331e: Status 404 returned error can't find the container with id f5fa3a8ae9e696cbd517f3ea73bbdaadd11f53ebe8079d1c38ec3f947a04331e
Sep 10 11:47:08.586916 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:08.586883    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6" event={"ID":"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d","Type":"ContainerStarted","Data":"5993d287d59c068119a9ae338eaa174d5360802ec59cb32e799b4677ba97b9fb"}
Sep 10 11:47:08.587328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:08.586966    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6" event={"ID":"9f38ad4d-05bc-4cea-a484-d8de6e9fc93d","Type":"ContainerStarted","Data":"f5fa3a8ae9e696cbd517f3ea73bbdaadd11f53ebe8079d1c38ec3f947a04331e"}
Sep 10 11:47:08.587328 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:08.586994    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:08.613107 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:08.613054    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6" podStartSLOduration=1.613037094 podStartE2EDuration="1.613037094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:47:08.6097351 +0000 UTC m=+1963.413802448" watchObservedRunningTime="2026-09-10 11:47:08.613037094 +0000 UTC m=+1963.417104441"
Sep 10 11:47:19.595543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:19.595462    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zxcz6"
Sep 10 11:47:26.102568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:26.102532    2585 scope.go:117] "RemoveContainer" containerID="0ad163b65116bc0fad956b5f87a5b1a9d777913a15bb8b2e4ab0d983bbc367b8"
Sep 10 11:47:31.096791 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:31.096747    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:47:37.177568 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:37.177537    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:47:44.974841 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:44.974802    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:47:51.950449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.950417    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:47:51.954207 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.954185    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.956505 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.956487    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f51e41c8-kserve-self-signed-certs\""
Sep 10 11:47:51.962973 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.962954    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:47:51.979059 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.979170 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979079    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.979170 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979134    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.979270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979186    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.979270 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979230    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft887\" (UniqueName: \"kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:51.979387 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:51.979285    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080605    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080651    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080683    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080806 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080716    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080971 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080871    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.080971 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.080940    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft887\" (UniqueName: \"kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.081075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.081048    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.081075 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.081070    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.081232 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.081210    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.082982 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.082960    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.083213 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.083194    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.093949 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.093923    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft887\" (UniqueName: \"kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887\") pod \"e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") " pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.263333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.263235    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:47:52.535422 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.535286    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:47:52.537627 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:47:52.537604    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fac4da4_aca4_4568_b1bf_83862adfc480.slice/crio-db90636953d7fc3633e9baff62817f8f76e05c377eaef6cb68e6a1f0f37d8b83 WatchSource:0}: Error finding container db90636953d7fc3633e9baff62817f8f76e05c377eaef6cb68e6a1f0f37d8b83: Status 404 returned error can't find the container with id db90636953d7fc3633e9baff62817f8f76e05c377eaef6cb68e6a1f0f37d8b83
Sep 10 11:47:52.792823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.792741    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" event={"ID":"1fac4da4-aca4-4568-b1bf-83862adfc480","Type":"ContainerStarted","Data":"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"}
Sep 10 11:47:52.792823 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.792779    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" event={"ID":"1fac4da4-aca4-4568-b1bf-83862adfc480","Type":"ContainerStarted","Data":"db90636953d7fc3633e9baff62817f8f76e05c377eaef6cb68e6a1f0f37d8b83"}
Sep 10 11:47:52.817996 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:52.817950    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" podStartSLOduration=1.817935442 podStartE2EDuration="1.817935442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:47:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:47:52.815660304 +0000 UTC m=+2007.619727652" watchObservedRunningTime="2026-09-10 11:47:52.817935442 +0000 UTC m=+2007.622002789"
Sep 10 11:47:54.575360 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:47:54.575326    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:02.263936 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:02.263901    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:02.263936 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:02.263945    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:02.276446 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:02.276418    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:02.855512 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:02.855480    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:10.175933 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:10.175897    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:26.121826 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:26.121796    2585 scope.go:117] "RemoveContainer" containerID="1a919e409f0984803cd61cb5a9274ad4df0aba31a0670af27f183f9a2e75c49b"
Sep 10 11:48:28.931230 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:28.931198    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:48:32.073409 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:32.073375    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:43.078685 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:43.078655    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:43.950510 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:43.950446    2585 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" podUID="1fac4da4-aca4-4568-b1bf-83862adfc480" containerName="main" containerID="cri-o://1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee" gracePeriod=45
Sep 10 11:48:44.187414 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.187388    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:44.339672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339584    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.339672 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339644    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.339909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339701    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.339909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339726    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ft887\" (UniqueName: \"kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.339909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339757    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.339909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339786    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs\") pod \"1fac4da4-aca4-4568-b1bf-83862adfc480\" (UID: \"1fac4da4-aca4-4568-b1bf-83862adfc480\") "
Sep 10 11:48:44.340117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339882    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home" (OuterVolumeSpecName: "home") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:48:44.340117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.339969    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:48:44.340117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.340034    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache" (OuterVolumeSpecName: "model-cache") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:48:44.340117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.340076    2585 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-tmp-dir\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:44.340117 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.340095    2585 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-home\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:44.341928 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.341897    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm" (OuterVolumeSpecName: "dshm") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 11:48:44.342040 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.341942    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 11:48:44.342040 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.341953    2585 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887" (OuterVolumeSpecName: "kube-api-access-ft887") pod "1fac4da4-aca4-4568-b1bf-83862adfc480" (UID: "1fac4da4-aca4-4568-b1bf-83862adfc480"). InnerVolumeSpecName "kube-api-access-ft887". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 11:48:44.441449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.441412    2585 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-dshm\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:44.441449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.441446    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft887\" (UniqueName: \"kubernetes.io/projected/1fac4da4-aca4-4568-b1bf-83862adfc480-kube-api-access-ft887\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:44.441449 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.441456    2585 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac4da4-aca4-4568-b1bf-83862adfc480-model-cache\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:44.441680 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:44.441466    2585 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac4da4-aca4-4568-b1bf-83862adfc480-tls-certs\") on node \"ip-10-0-132-86.ec2.internal\" DevicePath \"\""
Sep 10 11:48:45.056236 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.056201    2585 generic.go:356] "Generic (PLEG): container finished" podID="1fac4da4-aca4-4568-b1bf-83862adfc480" containerID="1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee" exitCode=0
Sep 10 11:48:45.056453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.056245    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"
Sep 10 11:48:45.056453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.056289    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" event={"ID":"1fac4da4-aca4-4568-b1bf-83862adfc480","Type":"ContainerDied","Data":"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"}
Sep 10 11:48:45.056453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.056356    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v" event={"ID":"1fac4da4-aca4-4568-b1bf-83862adfc480","Type":"ContainerDied","Data":"db90636953d7fc3633e9baff62817f8f76e05c377eaef6cb68e6a1f0f37d8b83"}
Sep 10 11:48:45.056453 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.056374    2585 scope.go:117] "RemoveContainer" containerID="1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"
Sep 10 11:48:45.064965 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.064949    2585 scope.go:117] "RemoveContainer" containerID="1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"
Sep 10 11:48:45.065263 ip-10-0-132-86 kubenswrapper[2585]: E0910 11:48:45.065245    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee\": container with ID starting with 1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee not found: ID does not exist" containerID="1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"
Sep 10 11:48:45.065360 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.065272    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee"} err="failed to get container status \"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee\": rpc error: code = NotFound desc = could not find container \"1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee\": container with ID starting with 1e886631bb55e0dd1333d1c5195e6089f01eecb269451d5a02f44d986664faee not found: ID does not exist"
Sep 10 11:48:45.080287 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.080265    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:48:45.084133 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.084113    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f51e41c8-kserve-8697879d75-5p75v"]
Sep 10 11:48:45.663099 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:45.663058    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fac4da4-aca4-4568-b1bf-83862adfc480" path="/var/lib/kubelet/pods/1fac4da4-aca4-4568-b1bf-83862adfc480/volumes"
Sep 10 11:48:50.475600 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:50.475506    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:58.462229 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.462196    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:48:58.819792 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.819715    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"]
Sep 10 11:48:58.820080 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.820069    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fac4da4-aca4-4568-b1bf-83862adfc480" containerName="main"
Sep 10 11:48:58.820127 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.820082    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fac4da4-aca4-4568-b1bf-83862adfc480" containerName="main"
Sep 10 11:48:58.820166 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.820148    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fac4da4-aca4-4568-b1bf-83862adfc480" containerName="main"
Sep 10 11:48:58.824658 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.824639    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.827681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.827655    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 11:48:58.827681 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.827677    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gkwvn\""
Sep 10 11:48:58.827862 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.827706    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 11:48:58.827862 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.827795    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 11:48:58.828046 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.828032    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 11:48:58.836045 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.836025    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"]
Sep 10 11:48:58.864145 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.864121    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-metrics-certs\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.864243 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.864170    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfsrr\" (UniqueName: \"kubernetes.io/projected/0261dba0-1a06-4b78-b281-d5f3680b4d65-kube-api-access-zfsrr\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.864243 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.864202    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-maas-api-tls\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.864358 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.864251    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0261dba0-1a06-4b78-b281-d5f3680b4d65-openshift-service-ca\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.965134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.965098    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfsrr\" (UniqueName: \"kubernetes.io/projected/0261dba0-1a06-4b78-b281-d5f3680b4d65-kube-api-access-zfsrr\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.965283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.965150    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-maas-api-tls\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.965283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.965211    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0261dba0-1a06-4b78-b281-d5f3680b4d65-openshift-service-ca\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.965283 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.965263    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-metrics-certs\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.965867 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.965835    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0261dba0-1a06-4b78-b281-d5f3680b4d65-openshift-service-ca\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.967609 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.967589    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-metrics-certs\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.967684 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.967632    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0261dba0-1a06-4b78-b281-d5f3680b4d65-maas-api-tls\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:58.973790 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:58.973772    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfsrr\" (UniqueName: \"kubernetes.io/projected/0261dba0-1a06-4b78-b281-d5f3680b4d65-kube-api-access-zfsrr\") pod \"maas-api-dc7dbf8c8-7scbl\" (UID: \"0261dba0-1a06-4b78-b281-d5f3680b4d65\") " pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:59.134265 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:59.134187    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:48:59.405728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:59.405658    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"]
Sep 10 11:48:59.408009 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:48:59.407979    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0261dba0_1a06_4b78_b281_d5f3680b4d65.slice/crio-ff5264733ed6ba41531e36761751930f696096a85286153e261fcccce3f1132b WatchSource:0}: Error finding container ff5264733ed6ba41531e36761751930f696096a85286153e261fcccce3f1132b: Status 404 returned error can't find the container with id ff5264733ed6ba41531e36761751930f696096a85286153e261fcccce3f1132b
Sep 10 11:48:59.409704 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:48:59.409684    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 11:49:00.128889 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:00.128840    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl" event={"ID":"0261dba0-1a06-4b78-b281-d5f3680b4d65","Type":"ContainerStarted","Data":"ff5264733ed6ba41531e36761751930f696096a85286153e261fcccce3f1132b"}
Sep 10 11:49:01.135261 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:01.135178    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl" event={"ID":"0261dba0-1a06-4b78-b281-d5f3680b4d65","Type":"ContainerStarted","Data":"4217412c77e98516c30832a82ed814ef888af5b218f4e25b92819619f59fd12e"}
Sep 10 11:49:01.135666 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:01.135342    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:49:01.153746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:01.153696    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl" podStartSLOduration=1.704265027 podStartE2EDuration="3.153679953s" totalImagesPullingTime="1.449414926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 11:48:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:49:01.152959428 +0000 UTC m=+2075.957026775" watchObservedRunningTime="2026-09-10 11:49:01.153679953 +0000 UTC m=+2075.957747303"
Sep 10 11:49:05.572793 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:05.572758    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:49:07.145499 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:07.145462    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-dc7dbf8c8-7scbl"
Sep 10 11:49:11.720212 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:11.720178    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:49:18.815808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:18.815773    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-7d7xq"]
Sep 10 11:49:25.776279 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:25.776239    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:49:25.781216 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:25.781194    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-c4glv_0bcdf0eb-13b7-4f26-b45d-1c3270e3d28b/ovn-acl-logging/0.log"
Sep 10 11:49:26.141661 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:26.141577    2585 scope.go:117] "RemoveContainer" containerID="ecb593967683e359873032176fd6e70b06c51ca3a06f949c94a043c03f55ca7d"
Sep 10 11:49:26.149719 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:26.149699    2585 scope.go:117] "RemoveContainer" containerID="2020abf5b955e18e23b5b7f92560c5d9d7b945f27002a50bdd1d53d2e9959114"
Sep 10 11:49:26.156906 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:26.156885    2585 scope.go:117] "RemoveContainer" containerID="5a671ccba34fd3557082e5d472dcf00204a2391b5afbb42781b79f704d59e45d"
Sep 10 11:49:27.306277 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:27.306251    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d6d6998d4-d2479_6e2b0daa-2309-427c-aab7-ebc194807f00/copy-manifests/0.log"
Sep 10 11:49:27.314894 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:27.314869    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d6d6998d4-d2479_6e2b0daa-2309-427c-aab7-ebc194807f00/manager/0.log"
Sep 10 11:49:27.951808 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:27.951778    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-dc7dbf8c8-7scbl_0261dba0-1a06-4b78-b281-d5f3680b4d65/maas-api/0.log"
Sep 10 11:49:28.181752 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:28.181718    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817330-b9dc5_62b19c43-957d-4dc0-8b03-20915a0039bd/cleanup/0.log"
Sep 10 11:49:29.521745 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:29.521716    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-652w5_0c9e4297-1c9e-4cdb-b16f-dd19348fcf41/manager/0.log"
Sep 10 11:49:29.782780 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:29.782703    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zxcz6_9f38ad4d-05bc-4cea-a484-d8de6e9fc93d/manager/0.log"
Sep 10 11:49:29.912776 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:29.912747    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-7d7xq_346e98ca-5281-468a-8dc9-5373094d614f/limitador/0.log"
Sep 10 11:49:30.819078 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:30.819045    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4449d5c7-r8559_2319856e-6de0-4223-8c4c-5e76f83a83df/kube-auth-proxy/0.log"
Sep 10 11:49:32.333543 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:32.333514    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9mdrl_a0218e53-3223-4948-932c-f03d9f27e617/main/0.log"
Sep 10 11:49:32.450244 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:32.450215    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-5nzl2_55d2e410-6bb8-4327-8fac-237a3593d850/main/0.log"
Sep 10 11:49:41.166650 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:41.166611    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bzrjs_60beda01-3d15-4bc8-9682-d5d86b363440/global-pull-secret-syncer/0.log"
Sep 10 11:49:41.410677 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:41.410643    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xxcfc_c14e1463-ba1c-4732-ad85-75536443232f/konnectivity-agent/0.log"
Sep 10 11:49:41.461259 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:41.461160    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-86.ec2.internal_9850f6ddb2c79a0bcf915bd203335015/haproxy/0.log"
Sep 10 11:49:46.220010 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:46.219957    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-652w5_0c9e4297-1c9e-4cdb-b16f-dd19348fcf41/manager/0.log"
Sep 10 11:49:46.301108 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:46.301076    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-zxcz6_9f38ad4d-05bc-4cea-a484-d8de6e9fc93d/manager/0.log"
Sep 10 11:49:46.337689 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:46.337658    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-7d7xq_346e98ca-5281-468a-8dc9-5373094d614f/limitador/0.log"
Sep 10 11:49:48.290633 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.290597    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-zrhzc_d4d70ccf-8fe4-413c-ae67-bbcc066f59b1/monitoring-plugin/0.log"
Sep 10 11:49:48.444080 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.444053    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9m68_d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb/node-exporter/0.log"
Sep 10 11:49:48.476121 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.476092    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9m68_d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb/kube-rbac-proxy/0.log"
Sep 10 11:49:48.503028 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.503000    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9m68_d5d9ea26-cf99-42a6-81f2-3c8c2c98b8fb/init-textfile/0.log"
Sep 10 11:49:48.748832 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.748802    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/prometheus/0.log"
Sep 10 11:49:48.779333 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.779286    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/config-reloader/0.log"
Sep 10 11:49:48.807178 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.807154    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/thanos-sidecar/0.log"
Sep 10 11:49:48.830869 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.830842    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/kube-rbac-proxy-web/0.log"
Sep 10 11:49:48.854227 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.854199    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/kube-rbac-proxy/0.log"
Sep 10 11:49:48.879089 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.879061    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/kube-rbac-proxy-thanos/0.log"
Sep 10 11:49:48.914471 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.914437    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0cd5e2f-36db-484d-8251-e90ef882a87a/init-config-reloader/0.log"
Sep 10 11:49:48.946113 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.946068    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pkkmr_4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2/prometheus-operator/0.log"
Sep 10 11:49:48.969245 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:48.969218    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-pkkmr_4e5b7cc8-7d7c-4394-b581-6a8b5d7fbfc2/kube-rbac-proxy/0.log"
Sep 10 11:49:49.001215 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.001133    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-vm4wr_5ee02f86-2fc0-4e89-a07b-27f2e7d18be3/prometheus-operator-admission-webhook/0.log"
Sep 10 11:49:49.172639 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.172604    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/thanos-query/0.log"
Sep 10 11:49:49.195937 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.195909    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/kube-rbac-proxy-web/0.log"
Sep 10 11:49:49.220613 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.220580    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/kube-rbac-proxy/0.log"
Sep 10 11:49:49.248966 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.248938    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/prom-label-proxy/0.log"
Sep 10 11:49:49.277519 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.277442    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/kube-rbac-proxy-rules/0.log"
Sep 10 11:49:49.304653 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:49.304620    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd57f5d8-6scl6_44447154-eee3-418c-83b0-3e99e0a2f7dd/kube-rbac-proxy-metrics/0.log"
Sep 10 11:49:51.438710 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.438677    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"]
Sep 10 11:49:51.441959 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.441938    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.444714 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.444688    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-79gdg\"/\"openshift-service-ca.crt\""
Sep 10 11:49:51.445134 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.445116    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-79gdg\"/\"kube-root-ca.crt\""
Sep 10 11:49:51.445233 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.445211    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-79gdg\"/\"default-dockercfg-rl99x\""
Sep 10 11:49:51.454700 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.454679    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"]
Sep 10 11:49:51.531909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.531872    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-podres\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.531909 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.531914    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-lib-modules\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.532167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.531994    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzh6x\" (UniqueName: \"kubernetes.io/projected/2fd70747-34a9-4aed-a0d8-3b38c18460b5-kube-api-access-pzh6x\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.532167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.532040    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-proc\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.532167 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.532089    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-sys\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633207    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-podres\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633262 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633267    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-lib-modules\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633296    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzh6x\" (UniqueName: \"kubernetes.io/projected/2fd70747-34a9-4aed-a0d8-3b38c18460b5-kube-api-access-pzh6x\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633356    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-proc\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633417    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-podres\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633426    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-sys\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633475    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-sys\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633523 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633491    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-proc\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.633720 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.633447    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2fd70747-34a9-4aed-a0d8-3b38c18460b5-lib-modules\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.646728 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.646698    2585 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzh6x\" (UniqueName: \"kubernetes.io/projected/2fd70747-34a9-4aed-a0d8-3b38c18460b5-kube-api-access-pzh6x\") pod \"perf-node-gather-daemonset-qpmv7\" (UID: \"2fd70747-34a9-4aed-a0d8-3b38c18460b5\") " pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:51.752616 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:51.752518    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:52.027295 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.027268    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"]
Sep 10 11:49:52.029918 ip-10-0-132-86 kubenswrapper[2585]: W0910 11:49:52.029887    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2fd70747_34a9_4aed_a0d8_3b38c18460b5.slice/crio-aa88c83132c0f9b27db3998f89263a4129ee00218b7370c075f2d4f31f48fd49 WatchSource:0}: Error finding container aa88c83132c0f9b27db3998f89263a4129ee00218b7370c075f2d4f31f48fd49: Status 404 returned error can't find the container with id aa88c83132c0f9b27db3998f89263a4129ee00218b7370c075f2d4f31f48fd49
Sep 10 11:49:52.091680 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.091655    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-zvmv9_defa4256-2d6a-44af-ad9b-510d8ae6c24d/download-server/0.log"
Sep 10 11:49:52.381828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.381735    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7" event={"ID":"2fd70747-34a9-4aed-a0d8-3b38c18460b5","Type":"ContainerStarted","Data":"f7c99bbb33f72e8929422dff5185a381d218516063a86a6671affa2ea035a4b2"}
Sep 10 11:49:52.381828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.381792    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7" event={"ID":"2fd70747-34a9-4aed-a0d8-3b38c18460b5","Type":"ContainerStarted","Data":"aa88c83132c0f9b27db3998f89263a4129ee00218b7370c075f2d4f31f48fd49"}
Sep 10 11:49:52.381828 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.381820    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:49:52.399594 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:52.399536    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7" podStartSLOduration=1.399517615 podStartE2EDuration="1.399517615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 11:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 11:49:52.397360805 +0000 UTC m=+2127.201428154" watchObservedRunningTime="2026-09-10 11:49:52.399517615 +0000 UTC m=+2127.203584963"
Sep 10 11:49:54.009038 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:54.009004    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6nm6f_a12913c6-6220-42c6-af23-16abe84a67c0/dns/0.log"
Sep 10 11:49:54.039260 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:54.039233    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6nm6f_a12913c6-6220-42c6-af23-16abe84a67c0/kube-rbac-proxy/0.log"
Sep 10 11:49:54.266125 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:54.266041    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qv7xl_ded2fd1b-1a87-4842-a072-82ee5c3fc4a2/dns-node-resolver/0.log"
Sep 10 11:49:54.888446 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:54.888419    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tf7ph_8967943b-d3b6-444f-a079-a06ed65e35c9/node-ca/0.log"
Sep 10 11:49:56.423766 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:56.423734    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4449d5c7-r8559_2319856e-6de0-4223-8c4c-5e76f83a83df/kube-auth-proxy/0.log"
Sep 10 11:49:57.533074 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:57.533037    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-682b2_41ae446d-970e-4fc8-8084-cff298f430c9/serve-healthcheck-canary/0.log"
Sep 10 11:49:58.183135 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:58.183100    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qppc_57ff60e0-f7d0-416f-9414-4e9822ea06a8/kube-rbac-proxy/0.log"
Sep 10 11:49:58.211759 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:58.211732    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qppc_57ff60e0-f7d0-416f-9414-4e9822ea06a8/exporter/0.log"
Sep 10 11:49:58.238889 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:58.238866    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9qppc_57ff60e0-f7d0-416f-9414-4e9822ea06a8/extractor/0.log"
Sep 10 11:49:58.397097 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:49:58.397070    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-79gdg/perf-node-gather-daemonset-qpmv7"
Sep 10 11:50:01.394746 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:01.394717    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d6d6998d4-d2479_6e2b0daa-2309-427c-aab7-ebc194807f00/manager/0.log"
Sep 10 11:50:01.443145 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:01.443118    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d6d6998d4-d2479_6e2b0daa-2309-427c-aab7-ebc194807f00/copy-manifests/0.log"
Sep 10 11:50:02.462358 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:02.462329    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4449d5c7-r8559_2319856e-6de0-4223-8c4c-5e76f83a83df/kube-auth-proxy/0.log"
Sep 10 11:50:03.111790 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:03.111711    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b89dcdb5f-4v9ls_a0b2f34c-43c1-4902-a101-05df37397122/manager/0.log"
Sep 10 11:50:04.686619 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:04.686591    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f4449d5c7-r8559_2319856e-6de0-4223-8c4c-5e76f83a83df/kube-auth-proxy/0.log"
Sep 10 11:50:08.294355 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:08.294324    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-ps9b7_fd08c3f5-61f0-4253-988a-3f5fa3ef80b8/migrator/0.log"
Sep 10 11:50:08.320055 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:08.320025    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-ps9b7_fd08c3f5-61f0-4253-988a-3f5fa3ef80b8/graceful-termination/0.log"
Sep 10 11:50:08.724026 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:08.723991    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pnftn_9b445af6-616b-4a61-aa56-e540042678a4/kube-storage-version-migrator-operator/1.log"
Sep 10 11:50:08.724833 ip-10-0-132-86 kubenswrapper[2585]: I0910 11:50:08.724817    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pnftn_9b445af6-616b-4a61-aa56-e540042678a4/kube-storage-version-migrator-operator/0.log"